Advertisement

Pilot Validation of the Tuberous Sclerosis-Associated Neuropsychiatric Disorders (TAND) Checklist

Open AccessPublished:October 16, 2014DOI:https://doi.org/10.1016/j.pediatrneurol.2014.10.006

      Abstract

      Background

      Tuberous sclerosis complex is a multisystem disorder that includes a range of tuberous sclerosis–associated neuropsychiatric disorders (TAND). The lifetime prevalence rates of TAND are very high; yet surveys suggest that the majority of individuals with tuberous sclerosis never receive appropriate assessment or treatment for TAND. To aid systematic enquiry, a TAND Checklist was developed. Here, we performed pilot validation of the TAND Checklist.

      Method

      Mixed methods were used across two stages. In stage 1, we gathered feedback on the Checklist from tuberous sclerosis “expert professionals” and “expert parents and caregivers.” The aim was to examine face and content validity. Stage 2 involved the administration of the refined TAND Checklist to 20 parents of individuals with tuberous sclerosis concurrently with four widely used validated rating scales, to examine external validity and obtain qualitative feedback on face-to-face administration of the TAND Checklist.

      Results

      Twenty professionals and 62 parents and caregivers from 28 countries participated in the pilot. The TAND Checklist demonstrated good face and content validity with high overall mean and median scores. Qualitative analysis highlighted concerns about the likely use of the TAND Checklist, suggesting that family members and individuals with tuberous sclerosis should drive usage. Stage 2 results showed moderate-to-very good external validity across TAND domain and key subdomains. Internal consistency of domains and subdomains was acceptable to very good. Ninety-three percent of all participants (93%) reported four or more lifetime TAND behavioral difficulties.

      Conclusion

      The pilot validation suggested that the TAND Checklist could provide a useful screening tool in clinical settings.

      Keywords

      Introduction

      Tuberous sclerosis complex (TSC) is a genetic disorder with multisystem involvement that can affect most organ systems.
      • Crino P.B.
      • Nathanson K.L.
      • Henske E.P.
      The tuberous sclerosis complex.
      • Curatolo P.
      • Bombardieri R.
      • Jozwiak S.
      Tuberous sclerosis.
      The physical manifestations include benign tumors in the heart, kidneys, lungs, skin, and brain. TSC is caused by mutations in either of two genes, the TSC1 gene (9q34)
      • van Slegtenhorst M.
      • de Hoogt R.
      • Herman H.
      • et al.
      Identification of the tuberous sclerosis gene TSC1 on chromosome 9q34.
      • Povey S.
      • Burley M.W.
      • Attwood J.
      • et al.
      Two loci for tuberous sclerosis: one on 9q34 and one on 16p13.
      or the TSC2 gene (16p13.3).
      The European Chromosome 16 Consortium
      Identification and characterization of the tuberous sclerosis gene on chromosome 16.
      • Povey S.
      • Burley M.W.
      • Attwood J.
      • et al.
      Two loci for tuberous sclerosis: one on 9q34 and one on 16p13.
      TSC has a birth incidence estimated around one in 6000.
      • Osborne J.P.
      • Fryer A.
      • Webb D.
      Epidemiology of tuberous sclerosis.
      • O'Callaghan F.J.K.
      • Shiell A.W.
      • Osborne J.P.
      • Martyn C.N.
      Prevalence of tuberous sclerosis estimated by capture-recapture analysis.
      • Thiele E.A.
      • Jozwiak S.
      Natural history of tuberous sclerosis complex and overview of manifestations.
      Appropriate management and coordination of medical specialist care are crucial across the lifespan of individuals with TSC to limit morbidity and mortality in this disease.
      • Krueger D.A.
      • Northrup H.
      • Roberds S.
      • et al.
      Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.

      The neuropsychiatry of TSC

      TSC is also associated with a vast range of neuropsychiatric disorders.
      • Gillberg I.C.
      • Gillberg C.
      • Ahlsen G.
      Autistic behavior and attention deficits in tuberous sclerosis: a population-based study.
      • Smalley S.L.
      • Burger F.
      • Smith M.
      Phenotypic variation of tuberous sclerosis in single extended kindred.
      • Prather P.
      • de Vries P.J.
      Behavioral and cognitive aspects of tuberous sclerosis complex.
      • de Vries P.J.
      Neurodevelopmental, psychiatric and cognitive aspects of tuberous sclerosis complex.
      At a behavioral level, difficulties include restless and impulsive behavior, high rates of aggression,
      • Kopp C.M.C.
      • Muzykewicz D.A.
      • Staley B.A.
      • Thiele E.A.
      • Pulsifer M.B.
      Behavior problems in children with tuberous sclerosis complex and parental stress.
      • de Vries P.J.
      • Hunt A.
      • Bolton P.F.
      The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
      • Hunt A.
      A comparison of the abilities, health and behaviour of 23 people with tuberous sclerosis at age 5 and as adults.
      • Eden K.E.
      • de Vries P.J.
      • Moss J.
      • Richards C.
      • Oliver C.
      Self-injury and aggression in tuberous sclerosis complex: cross-syndrome comparison and associated risk markers.
      temper tantrums,
      • de Vries P.J.
      • Hunt A.
      • Bolton P.F.
      The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
      and self-injury.
      • de Vries P.J.
      • Hunt A.
      • Bolton P.F.
      The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
      • Hunt A.
      A comparison of the abilities, health and behaviour of 23 people with tuberous sclerosis at age 5 and as adults.
      • Staley B.A.
      • Montenegro M.
      • Major P.
      • et al.
      Self-injurious behavior and tuberous sclerosis complex: frequency and possible associations in a population of 257 patients.
      • Eden K.E.
      • de Vries P.J.
      • Moss J.
      • Richards C.
      • Oliver C.
      Self-injury and aggression in tuberous sclerosis complex: cross-syndrome comparison and associated risk markers.
      At the psychiatric level, developmental disorders, including autism spectrum disorders (ASD, 40-50%)
      • Bolton P.F.
      • Park R.J.
      • Higgins J.N.P.
      • Griffiths P.D.
      • Pickles A.
      Neuro-epileptic determinants of autism spectrum disorders in tuberous sclerosis complex.
      and attention deficit hyperactivity disorder (ADHD, 30-50%), are commonly observed.
      • Gillberg I.C.
      • Gillberg C.
      • Ahlsen G.
      Autistic behavior and attention deficits in tuberous sclerosis: a population-based study.
      • Prather P.
      • de Vries P.J.
      Behavioral and cognitive aspects of tuberous sclerosis complex.
      • Muzykewicz D.A.
      • Newberry P.
      • Danforth N.
      • Halpern E.F.
      • Thiele E.A.
      Psychiatric comorbid conditions in a clinic population of 241 patients with tuberous sclerosis complex.
      High rates of depression and anxiety disorder have also been documented.
      • de Vries P.J.
      • Hunt A.
      • Bolton P.F.
      The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
      • Muzykewicz D.A.
      • Newberry P.
      • Danforth N.
      • Halpern E.F.
      • Thiele E.A.
      Psychiatric comorbid conditions in a clinic population of 241 patients with tuberous sclerosis complex.
      • Lewis J.C.
      • Thomas H.V.
      • Murphy K.C.
      • Sampson J.R.
      Genotype and psychological phenotype in tuberous sclerosis.
      • Raznahan A.
      • Joinson C.
      • O’Callaghan F.O.
      • Osborne J.P.
      • Bolton P.F.
      Psychopathology in tuberous sclerosis: and overview and findings in a population-based sample of adults with tuberous sclerosis.
      • de Vries P.J.
      Targeted treatments for cognitive and neurodevelopmental disorders and tuberous sclerosis complex.
      At the intellectual level, approximately 50% of individuals with TSC have normal intellectual abilities, and others have varying degrees of intellectual disability (ID).
      • de Vries P.J.
      Neurodevelopmental, psychiatric and cognitive aspects of tuberous sclerosis complex.
      • Joinson C.
      • O’Callaghan F.J.
      • Osborne J.P.
      • Martyn C.
      • Harris T.
      • Bolton P.
      Learning difficulties and epilepsy in an epidemiological sample of individuals with tuberous sclerosis complex.
      • de Vries P.J.
      • Prather P.
      The tuberous sclerosis complex (TSC).
      At the academic level, many school-aged children with TSC have academic difficulties, for instance, in mathematics, reading, writing, and spelling.
      • de Vries P.J.
      Neurodevelopmental, psychiatric and cognitive aspects of tuberous sclerosis complex.
      At the neuropsychologic level a range of neuropsychologic deficits are also observed. These include difficulties with executive, attentional, memory, and language skills.
      • Prather P.
      • de Vries P.J.
      Behavioral and cognitive aspects of tuberous sclerosis complex.
      • de Vries P.J.
      Neurodevelopmental, psychiatric and cognitive aspects of tuberous sclerosis complex.
      • Harrison J.E.
      • O’Callaghan F.J.
      • Hancock E.
      • Osborne J.
      • Bolton P.F.
      Cognitive deficits in normally intelligent patients with tuberous sclerosis.

      de Vries, PJ. (2002). The psychopathologies of attention on tuberous sclerosis. PhD Thesis, University of Cambridge, Cambridge, UK.

      • Ridler K.
      • Suckling J.
      • Higgins N.J.
      • et al.
      Neuroanatomical correlates of memory deficits in tuberous sclerosis complex.
      • de Vries P.J.
      • Gardiner J.
      • Bolton P.F.
      Neuropsychological attention deficits in tuberous sclerosis complex (TSC).
      • Tierney K.M.
      • McCartney D.L.
      • Serfontein J.R.
      • de Vries P.J.
      Neuropsychological attention skills and related behaviours in adults with tuberous sclerosis complex.
      At the psychosocial level, there is growing evidence of the impact of TSC on, for instance, self-esteem, family stress, and parental relationships.
      • Whittemore V.H.
      • Lewis J.
      Impact of TSC on the family and genetic counselling Issues.
      Each individual with TSC will exhibit their own unique combination of strengths and weaknesses, and this profile may change over time. Taken together, the majority of individuals with TSC will have some neuropsychiatric problems in their lifetime, with lifetime prevalence rates in the region of 90%.
      • de Vries P.J.
      • Whittemore V.H.
      • Leclezio L.
      • et al.
      Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.

      TSC-associated neuropsychiatric disorders (TAND)

      In 2010, a survey of members of the Tuberous Sclerosis Association in the United Kingdom indicated that only 18% of individuals with TSC had ever received an assessment or treatment for neuropsychiatric disorders (personal communication P.J.d.V.). These results suggested a treatment gap of around 70%. At the 2012 International TSC Consensus Conference,
      • Krueger D.A.
      • Northrup H.
      • Roberds S.
      • et al.
      Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.
      the Neuropsychiatry panel expressed concern about the enormous treatment gap and about the confusion of terminology across different levels of investigation of the biopsychosocial aspects of TSC. The panel decided to coin the term TAND, TSC-Associated Neuropsychiatric Disorders, as an umbrella term to refer to all the levels of investigation listed previously and agreed to generate a TAND Checklist as a potential screening tool.
      • Krueger D.A.
      • Northrup H.
      • Roberds S.
      • et al.
      Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.
      • de Vries P.J.
      • Whittemore V.H.
      • Leclezio L.
      • et al.
      Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.
      The panel did not want to develop a “diagnostic” tool, but rather a screening “Checklist” to guide health-care teams in a systematic enquiry of the current behavioral, psychiatric, intellectual, academic, neuropsychologic, and psychosocial difficulties of the individual with TSC. Details of the conceptualization of TAND and the TAND Checklist are presented in de Vries et al.
      • de Vries P.J.
      • Whittemore V.H.
      • Leclezio L.
      • et al.
      Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.
      Checklists are aimed at reducing errors of omission and are generally easy to administer and understand.
      • Scriven M.
      Logic of evaluation.
      Although numerous standardized tools existed for screening and diagnosis of a range of neuropsychiatric disorders, many of these tools have not been validated across all ages and developmental levels, most are not routinely available at clinics, and where they are used, tools are typically copyrighted with a charge per use.

      The TAND Checklist

      One of the goals of the Neuropsychiatry Panel was therefore to develop a simple TSC Checklist that would be globally and freely available to all clinicians and families. The TAND Checklist
      • de Vries P.J.
      • Whittemore V.H.
      • Leclezio L.
      • et al.
      Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.
      includes an item on basic developmental milestones (Question 1), one on current level of functioning (Question 2), a behavioral item with 19 YES/NO questions about behaviors of concern (Question 3), a psychiatric item listing high frequency mental health diagnoses seen in TSC (Question 4), and items on intellectual disability (Question 5), academic skills (Question 6), neuropsychologic skills (Question 7), and psychosocial functioning (Question 8). The TAND Checklist also includes a parent, caregiver or self-rating of the impact of TAND (Question 9) and a similar item where the health-care professional who completes the TAND Checklist with the person provides an overall TAND impact score (Question 12). Items 10 and 11 allow for prioritization or addition of extra concerns.
      As part of the development of the TAND Checklist, it was important that it be deemed to have face validity (observed by professionals and families as capturing the essential and important aspects of concern), content validity (judged by experts to cover the range of neuropsychiatric concerns of relevance to TSC), and transferability (the ability of the tool to be used across different settings by different people). Here, we performed pilot validation of the TAND Checklist with the aim of evaluating the face, content, and subsequent validity as well as internal consistency and external validity of the tool.

      Methods

      The pilot study was conducted in two stages using mixed methodology. In Stage 1, quantitative and qualitative feedback was collected on the draft TAND Checklist from two expert groups, a multidisciplinary panel of international TSC experts (referred to as the “expert professional” group) and an international panel of user and caregiver representatives (referred to as the “expert parent and caregiver” group). Experts were provided with either an electronic copy or a paper version of the TAND Checklist in English and an Expert Feedback Form in English. The Expert Feedback Form consisted of five quantitative and open-ended qualitative items to capture aspects of comprehensiveness, clarity, ease of use, applicability, and subsequent validity. Stage 1 data were used to examine internal consistency of the TAND Checklist.
      In Stage 2, the TAND Checklist, modified based on feedback from Stage 1, was administered to parents and caregivers of individuals with TSC in Cape Town, South Africa. After completion of the TAND Checklist with a research psychologist (L.L.), parents and caregivers were asked to complete the Expert Feedback Form and were then asked to complete four well-established and widely used rating scale measures: the Strengths and Difficulties Questionnaire (SDQ),
      • Goodman R.
      Psychometric properties of the strengths and difficulties questionnaire.
      a widely used behavioral screening questionnaire; the Social Communication Questionnaire (SCQ),
      • Berument S.K.
      • Rutter M.
      • Lord C.
      • Pickles A.
      • Bailey A.
      Autism screening questionnaire: diagnostic validity.
      a secondary screening tool for autism spectrum disorder; the Behaviour Rating Inventory of Executive Functions (BRIEF), developed to quantify behavioral manifestations associated with executive functioning in children, adolescents, and adults
      • Gioia G.A.
      • Isquith P.K.
      • Guy S.C.
      • Kenworthy L.
      Behavior rating inventory of executive function.
      ; and the Wessex Scale,
      • Kushlick A.
      • Blunden R.
      • Cox G.R.
      A method of rating behaviour characteristics for use in large scale surveys of mental handicap.
      a measure of adaptive behavior as proxy measure of intellectual disability (ID).

      Study participants

      “Expert Professionals” were recruited in collaboration with the Tuberous Sclerosis Alliance to represent wide ranging areas of expertise relevant to TSC. Snowball sampling was used where TSC expert professionals were asked to recommend other TSC expert professionals for participation until the desired number of responses (n = 20) was received. “Expert parents and caregivers” were recruited through two mechanisms. The first group consisted of parents, carers and individual members of Australasian Tuberous Sclerosis Society. The second group was representatives of Tuberous Sclerosis Complex International, a global network of TSC parent, user and caregiver organizations. All Tuberous Sclerosis Complex International representatives were invited to participate. Study participants for Stage 2 were recruited through the Red Cross War Memorial Children's Hospital TSC clinic in Cape Town, South Africa. Potential participants had to meet definite criteria for TSC
      • Roach E.S.
      • Gomez M.R.
      • Northrup H.
      Tuberous sclerosis complex consensus conference: revised clinical diagnostic criteria.
      • Northrup H.
      • Krueger D.A.
      • Roberds S.
      • et al.
      Tuberous Sclerosis Complex Diagnostic Criteria Update: Recommendations of the 2012 International Tuberous Sclerosis Complex Consensus Conference.
      and had to have a parent or caregiver who could complete the research questionnaires and interview in English. The research team continued to recruit until 20 participants were identified. All participants in this study were required to understand English and only an English version of the TAND Checklist was used in Stages 1 and 2.

      Research ethics

      The study was conducted in compliance with the Declaration of Helsinki. The protocol was peer reviewed in the Department of Psychiatry at the University of Cape Town and submitted for ethical approval at the Faculty of Health Sciences, Human Research Ethics Committee (Ethics Ref 200/2013). All participants received information about the study and provided written informed consent. No young people (<18 years) or individuals with intellectual disability directly participated in the study.

      Data analysis

      For the TAND Checklist, individual items were scored as simple yes or no responses. Selected items were grouped together to form domains and subdomains for the purpose of analysis along with the four external assessment tools total and subscale scores. For pilot validation, we used the Behavioral domain (Question 3, subdomains included “hyperactivity” and “social communication”), Intellectual ability domain (Question 5), the Academic domain (Question 6), Neuropsychologic domain (Question 7, subdomain “executive skills”), Psychosocial domain (Question 8), and the two Impact scores (Questions 9 and 12).
      Standard scoring methods were used for the SDQ, SCQ, and BRIEF tools. No standardized scoring procedures for the Wessex have been published to date. For the purpose of this study, consensus judgment scores of intellectual ability based on information provided in the Wessex questionnaire were generated by two of the authors (L.L., P.J.d.V.), blind to the TAND Checklist information.
      Data were analyzed using IBM SPSS Statistics for Macintosh, Version 21.0 (IBM, Armonk, NY). Quantitative data analysis was performed using nonparametric tests given the relatively small sample size. Item-by-item analysis was examined by applying the Mann-Whitney test, and the chi-square test was used for dichotomous variables. For interpretation of Spearman ρ values generated by correlations, standard convention was used (Table 1). Internal consistency of the TAND Checklist was examined by applying Cronbach α coefficient. Interpretation of Cronbach α values generated by correlations is listed in Table 2.
      Table 1Interpretation Table of Spearman Rank-Order Correlation Coefficients
      (Adapted From Dancey and Reidy, 2004)
      • Dancey C.
      • Reidy J.
      Statistics without maths for psychology: using SPSS for windows.
      Spearman ρCorrelation
      ≥0.70Very strong relationship
      0.40-0.69Strong relationship
      0.30-0.39Moderate relationship
      0.20-0.29Weak relationship
      0.01-0.19No or negligible relationship
      This descriptor applies to both positive and negative relationships.
      Table 2Interpretation Table of Cronbach α Correlation Coefficients
      (Based on Kline, 1999)
      • Kline P.
      The handbook of psychological testing.
      Cronbach αInternal consistency
      α ≥ 0.9Excellent (high stakes testing)
      0.7 ≤ α < 0.9Good (low stakes testing)
      0.6 ≤ α < 0.7Acceptable
      0.5 ≤ α < 0.6Poor
      α < 0.5Unacceptable
      Qualitative data were analyzed using summative content analysis,
      • Hsieh H.F.
      • Shannon S.E.
      Three approaches to qualitative content analysis.
      which consisted of counting and comparing keywords and concepts followed by interpretation of the underlying constructs.

      Results

      Stage 1 results—expert review of the TAND Checklist

      Twenty expert feedback forms were returned by expert professionals. Sixty-five percent (65% or 13 of 20) completed the quantitative items, and 85% (17 of 20) provided both quantitative and qualitative feedback. All data were used for analysis. Forty-two (42) parent or caregiver expert feedback forms were returned. One hundred percent completed the quantitative items, and 81% (34 of 42) completed both quantitative and qualitative questions.

      Stage 1 quantitative feedback

      The Expert Feedback Form asked respondents to rate five questions on a Likert scale from 0 to 5 with 5 as the highest score and allowed for comments on each question. Given the relatively small sample size, means, median, and standard deviations are presented (Table 3).
      Table 3Stage 1 Quantitative Expert Feedback About the TAND Checklist
      ItemExpert Professional, Mean (SD), N = 13Expert Professional, MedianExpert Parent and Caregiver, Mean (SD), N = 42Expert Parent and Caregiver, MedianMann-Whitney U
      Statistical comparison between expert professional and expert parent scores.
      P Value
      Statistical comparison between expert professional and expert parent scores.
      Comprehensiveness4.62 (0.87)54.60 (0.80)52600.741
      Clarity4.31 (0.85)44.48 (0.78)52290.466
      Ease of use4.31 (0.95)54.56 (0.79)5214.50.332
      Clinician usage3.77 (0.83)43.43 (1.04)3178.50.232
      Subsequent referral4.15 (0.69)44.11 (0.84)4236.50.925
      Abbreviations:
      SD = Standard deviation
      TAND = Tuberous sclerosis complex-associated neuropsychiatric disorders
      Key to scores: 1 = “not at all”;
      5 = “very much.”
      Statistical comparison between expert professional and expert parent scores.
      Feedback from expert professional participants showed that the median score for Items 1 and 2 (“comprehensiveness” and “clarity”) were 5 out of a maximum 5 and Items 3-5 (“ease of use,” “likelihood of clinician use,” “likelihood of next step evaluation, treatment or referral”) were scored 4 of 5.
      Expert parents and caregivers had a median score of 5 on Items 1-3 relating to comprehensiveness, clarity, and ease of use. Item 4 (“How likely do you think clinicians are to use the Checklist?”) had a median score of 3. Item 5 (“How likely is the Checklist to encourage clinicians to pursue further neuropsychiatric evaluation or referral to relevant specialists?”) had a median score of 4. Statistical comparison between expert professional and expert parent scores showed no significant differences (Table 3).

      Stage 1 qualitative feedback

      For qualitative analysis, all comments made by the expert professionals and expert parents (n = 69) were used. Summative analysis revealed six key themes (Fig 1). The first theme related to administration, such as where the TAND Checklist should be administered and by whom. The second theme that emerged centered around intellectual ability. Respondents felt it was important to establish the level of intellectual ability of a participant at the start of the TAND Checklist as it may influence administration of the remaining questions. Both expert professionals and expert parents and caregivers suggested including examples that would make it easier for parents to understand specific technical or medical terms such as “visuo-spatial skills.” There was a total of 22 comments on missing items where experts suggested the inclusion of additional items. Nine comments proposed that the TAND Checklist also be used for other purposes such as research or training. The last theme that emerged, overwhelmingly from the parent group (13 comments), highlighted the need for parents to drive clinical usage of the TAND Checklist. Feedback from Stage 1 was used to revise the TAND Checklist, and the revised TAND Checklist was used in Stage 2 of the study.
      Figure thumbnail gr1
      Figure 1Frequency distribution of qualitative responses from Expert groups across 6 themes. (The color version of this figure is available in the online edition.)

      TAND Checklist internal consistency

      The total number of behavioral items (Question 3) on the TAND Checklist showed good internal consistency (α = 0.884). The hyperactivity subdomain items (Question 3n-3q) also generated a high Cronbach α (α = 0.751), and the social communication subdomain (Question 3h-3m) showed an acceptable level of internal consistency (α = 0.682). The four components in the academic domain (Question 6) showed excellent internal consistency (α = 0.954). Both the overall neuropsychologic domain items (Question 7) and executive function subdomain items (Question 7b-7e) showed good internal consistency (overall α = 0.783, executive subdomain α = 0.792). Internal consistency of the psychosocial domain (Question 8) was relatively poor (α = 0.365).

      Stage 2 results—face-to-face administration of the TAND Checklist and external validation

      A total of 20 parents, caregivers, or individuals with TSC were recruited for Stage 2. The mean age of our TSC population of 20 patients was 14.25 years (range 3-42 years). The gender ratio was 12:8 male to female.

      Stage 2 quantitative feedback

      The median scores assigned across the five questions were 5 for Items 1, 2, and 5 and 4 for Items 3 and 4. Scores on Items 1 and 3 ranged between 3 and 5, Item 2 was scored either 4 or 5, and Items 4 and 5 had a slightly broader range between 2 and 5 (Table 4). To determine whether live administration may have led to different perceptions of the TAND Checklist, results from parents in Stage 2 was compared with parents and carers in Stage 1. No statistical differences were identified across four of the five items between scores on Stage 2 live administration and Stage 1 expert parents and caregivers except Item 2 (clarity) that was rated significantly higher in the Stage 2 live administration group (Mann-Whitney U = 249, P = 0.003; Table 5).
      Table 4Stage 2 Quantitative Feedback About the TAND Checklist Following Face-to-Face Administration With Stage 2 Participants
      ItemMedian, N = 20Mean (SD)
      Comprehensiveness54.8 (0.52)
      Clarity54.95 (0.22)
      Ease of use44.45 (0.69)
      Clinical usage43.65 (0.14)
      Subsequent referral54.4 (0.88)
      Abbreviations:
      SD = Standard deviation
      TAND = Tuberous sclerosis complex-associated neuropsychiatric disorders
      Key to scores: 1 = “not at all”
      5 = “very much.”
      Table 5Comparison of Pen-and-Paper versus Live Administration of the TAND Checklist to Expert Parents and Caregivers
      ItemStage 1 vs Stage 2 Mann-Whitney UP Value
      Comprehensiveness3640.259
      Clarity2490.003
      Ease of use341.50.362
      Clinical usage311.50.482
      Subsequent referral292.50.162
      Abbreviation:
      TAND = Tuberous sclerosis associated neuropsychiatric disorders

      Stage 2 qualitative feedback

      Nine qualitative comments were received from parents during Stage 2. These included contemporaneously documented comments during administration. Comments included possible missing items, but were mainly around future concerns and psycho-educational questions about TSC. Families reported the process of participation as very positive and validating.

      External validation

      External validation aimed to compare domains and subdomains of the TAND Checklist with relevant well-validated external tools. Figure 2 illustrates the correlation between the TAND Checklist behavioral domain total score (Question 3a-3s) and the total difficulties score on the SDQ.
      Figure thumbnail gr2
      Figure 2Correlation between the TAND Checklist Total Behavior Score (maximum score = 19) and the SDQ Total Difficulties Score (maximum score = 40; Rho = 0.81; P < 0.001). (The color version of this figure is available in the online edition.)
      Results reveal a strong positive correlation (ρ = 0.81, P < 0.001). To examine hyperactivity-related behaviors, the TAND Checklist hyperactivity subdomain items (Question 3n-3q) were plotted against the hyperactivity or inattention domain items of the SDQ. Results demonstrated a strong correlation (ρ = 0.77, P < 0.001). The TAND Checklist social communication subdomain items or score (Question 3h-3m) and the total scores on the SCQ show a strong linear correlation (ρ = 0.70, P = 0.002). The SDQ prosocial domain is a measure of positive or prosocial behaviors, predicted to correlate inversely with social communication difficulties. Results confirmed a strong negative correlation (ρ = −0.65, P = 0.002) between the prosocial domain of the SDQ and the TAND social communication subdomain score. In Question 5, parents were asked about intellectual disability in their child or family member. Parental judgment of the presence or absence of ID was compared with researcher judgment based on the Wessex questionnaire scores. Cross-tabulation of findings is illustrated in Fig 3 (Fisher exact test, P < 0.001). The two-by-two contingency table showed a significant association between the two classifications (Fisher exact test, P < 0.001).
      Figure thumbnail gr3
      Figure 3Cross-tabulation of the relationship between parental judgement of intellectual ability and clinician rating based on the Wessex scale (Fisher's Exact P < 0.001).
      The neuropsychologic domain score (Question 7a-7f) was plotted against the domain scores of the BRIEF. Results revealed a strong positive correlation between with the Global Executive Score (ρ = 0.79, P < 0.001) and the BRIEF Behavior Rating Index (BRI) score (ρ = 0.74, P = 0.001) and moderate correlation with the BRIEF Metacognition Index (MI; ρ = 0.59, P = 0.016). Given the fact that the TAND Checklist neuropsychologic domain included a number of executive skills (specifically measured in the BRIEF), it was important to examine executive skills specifically. The TAND Checklist executive skills subdomain scores (Question 7b-7e) showed strong correlation with the BRIEF Global Executive Score (ρ = 0.79, P < 0.001), the BRI score (ρ = 0.75, P = 0.001), and the MI score (ρ = 0.65, P = 0.006). The correlation between the TAND executive subdomain and BRIEF BRI domain is shown in Fig 4.
      Figure thumbnail gr4
      Figure 4Correlation between the TAND Executive Function Score and BRIEF Behaviour Rating Index (Rho = 0.75; P = 0.001). (The color version of this figure is available in the online edition.)
      No external tools of academic skills were included in this study. However, we predicted that individuals with a lower Wessex score, suggesting ID, would have higher rates of academic difficulties reported in their TAND Checklist. Eighty percent (16 of 20) of participants were of school going age or above and could be examined for scholastic difficulties. The TAND Checklist identified seven individuals with academic difficulties of whom six were judged to have ID as based on the Wessex Scale.
      Administration of the TAND Checklist took ∼10 minutes, and the duration of Stage 2 data collection was between 45 minutes and 1.5 hours.

      How many participants had lifetime TAND difficulties reported?

      The TSC literature summarized in the introduction provided rates of difficulties across groups of individuals, for instance, to report that 40% of children with TSC had anxiety symptoms or that 57% had temper tantrums.
      • de Vries P.J.
      • Hunt A.
      • Bolton P.F.
      The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
      However, there were no data to indicate what proportion of individuals with TSC had one or more of these TAND behavioral challenges as a marker of lifetime rates of TAND difficulties. We therefore calculated the number of participants (of the total n = 62) who had a lifetime report of one or more TAND behavioral difficulties endorsed. One hundred percent of participants had one or more lifetime reported TAND behavioral difficulties, 97% had two or more difficulties, 93% had four or more difficulties, and 89% had six or more lifetime behavioral difficulties.

      Discussion

      Results from this study demonstrated high scores across the main areas of face and content validity examined. Experts from 28 countries participated in Stage 1 suggesting that the TAND Checklist has broad and global face and content validity. The many helpful suggestions from experts were incorporated into the revised version of the TAND Checklist, such as addition of a developmental section at the start of the TAND Checklist to provide an overview of the functional ability level of the participant. Results from Item 4 on the Expert Feedback Form (“clinical usage”) indicated hesitation as to whether clinical teams would use the TAND Checklist in practice. It is possible that there may have been concern regarding the time requirements to complete the tool in the context of a busy clinic schedule or that experts did not feel that they would have the necessary competence to complete the TAND Checklist with families. It was therefore interesting to observe a strong theme from expert parents about the need for parents and families to take ownership and drive usage of the TAND Checklist.
      No statistical differences were observed between responses of expert professionals and expert parents in Stage 1. It was therefore interesting to observe a statistically significant difference between Stage 1 expert parents or caregivers and Stage 2 live administration participants regarding clarity. Results suggest that face-to-face administration of the TAND Checklist led to increased clarity, providing good support for the face-to-face approach when using the TAND Checklist.
      Examination of internal consistency suggested that the TAND Checklist has acceptable-to-excellent internal consistency within the domains and subdomains measured. The items from the psychosocial domain did not appear to have good internal consistency. On closer inspection, the three elements of this item include intra- and interpersonal factors (self-esteem, family stress, and parental relationship stress), where high internal consistency may not be expected. We suggest that the psychosocial domain should therefore be used simply as an introduction to a conversation about this important level of investigation.
      One of the main objectives of the study was to investigate external validity of the TAND Checklist domain and subdomains. The behavioral domain items of the TAND Checklist correlated very strongly with the total difficulties score of the SDQ, suggesting that the TAND behavioral question may be helpful at identifying a range of behavioral difficulties that may underlie a range of psychopathologies as screened for using the SDQ. Results within the subdomain of hyperactivity also showed strong correlation between items associated with hyperactivity in the TAND Checklist and the total hyperactivity or inattention score produced by the SDQ assessment tool, suggesting that endorsement of the hyperactivity items on the TAND Checklist should raise the clinical suspicion of ADHD or an attention-related disorder. The TAND Checklist social communication subdomain constructs correlated strongly with items from the SCQ, highlighting behaviors associated with ASDs. Findings suggested that these items may be very useful markers of risk for ASD, which is known to have a very high prevalence in TSC. Overall, results from the behavioral domain suggested that ADHD- and ASD-related behaviors, two key developmental challenges in TSC, may usefully be identified through the TAND Checklist.
      There was a moderate correlation between the level of intellectual ability as perceived by parents and researcher judgment based on the Wessex scale. Results suggest that parental perception of intellectual development is generally reasonably accurate. Given the multicomponential nature of intelligence, all individuals with TSC are recommended to have a formal assessment of their intellectual strengths and weaknesses at key developmental time points.
      • Krueger D.A.
      • Northrup H.
      • Roberds S.
      • et al.
      Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.
      At the neuropsychologic level, the TAND Checklist showed very strong correlation with the BRIEF. There were strong correlations when comparing the total TAND neuropsychologic score with the global executive score and BRI of the BRIEF, suggesting that the TAND Checklist may be useful to highlight broad overall neuropsychologic concerns and behavior-related difficulties such as inhibition, shifting between tasks, or emotional control. The moderate correlation observed between the TAND total score and the MI of the BRIEF suggested that the TAND Checklist did not fully capture the finer constructs identified by the MI including initiation, working memory, planning or organizing, and monitoring skills. It was very encouraging that the TAND Checklist executive function subdomain correlated strongly with all three subscales of the BRIEF. Taken together, results suggest that the TAND Checklist may be very helpful in identifying individuals at risk of potential neuropsychologic and, in particular, executive difficulties that would benefit from further evaluation and intervention.
      The striking finding that almost 90% of participants in the study had six or more lifetime TAND behavioral difficulties underlined why TAND is such a crucial clinical domain to consider in real life. Further investigations of the lifetime rates across TAND levels of investigation may provide extremely helpful information.

      Limitations of the study

      In spite of the positive initial findings of this pilot study, it is important to consider potential limitations. This study did not examine reliability of the TAND Checklist such as inter-rater or test-retest reliability. It might be very helpful to examine inter-rater reliability, in particular, to see if relatively nonexpert clinicians will get similar scores to very experienced TSC clinicians. We predict that the quality of information collected through the TAND Checklist will most strongly depend on the quality of the rapport between the interviewer and interviewee. Test-retest reliability is often examined for questionnaires. It is not clear how useful this would be for a TAND Checklist given that new neuropsychiatric manifestations may present over the course of a few weeks to months, thus reducing the likelihood of high stability of measurement. It was outside the scope of this study to examine sensitivity and specificity of the tool. As raised in the introduction, the purpose of the TAND Checklist was not to generate a “diagnostic tool” with thresholds or “cut-off values” for disorders (see also details of the conceptualization of TAND and the TAND Checklist
      • de Vries P.J.
      • Whittemore V.H.
      • Leclezio L.
      • et al.
      Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.
      ). For this reason, sensitivity and specificity were not the key considerations in this pilot validation. Further evaluation of other psychometric properties of the TAND Checklist may be natural next steps.

      Future directions

      Further research is required to replicate and extend investigation of the psychometric properties of the TAND Checklist. Further subsequent validity research studies will help to ascertain whether annual screening of TAND will address the treatment gap of neuropsychiatric disorders. Families and clinicians may also benefit from a tool kit containing “next step, self-help” information and access to resources, especially in low- and middle-income countries and other low resource environments where access to highly specialized clinicians and TSC centers is limited.
      We deliberately chose to perform pilot validation in South Africa, a middle-income country, where we expected baseline knowledge not to be very high, and where levels of academic and socioeconomic status and competence in English would be quite variable. We argued that the tool needed to be such that it would be easily accessible to individuals from low- and middle countries. It was therefore very encouraging to observe that one simple tool that takes approximately 10 minutes to complete captured information relevant to TSC in a way that correlated very well with four external tools.

      Conclusion

      In this research project, we performed the first evaluation of the TAND Checklist, a newly developed, freely available tool to screen for TAND. Results suggested that overall the TAND Checklist was deemed to be a good tool to identify possible neuropsychiatric difficulties. Qualitative feedback provided information for minor improvements to the TAND Checklist and raised the importance of families leading the use of the TAND Checklist in partnership with their health-care teams. We suggest that the TAND Checklist may be a helpful tool for annual screening of TAND, as recommended at the 2012 International Consensus Conference.
      • Krueger D.A.
      • Northrup H.
      • Roberds S.
      • et al.
      Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.
      We would like to thank Dr. Birgit Schlegel, Prof. Jo Wilmshurst, and Dr. Edward Kija, from the Department of Paediatrics, Red Cross War Memorial Children's Hospital, University of Cape Town, South Africa, for help with identification and recruitment of participants for Stage 2 of the study.
      We are grateful to all expert professionals, expert parents, caregivers, and individuals with TSC who participated in this study. A particular thanks to the Australasian Tuberous Sclerosis Society and Tuberous Sclerosis International for support. The study was supported by funding from the National Research Foundation , Struengmann Fund , University of Cape Town , and the Tuberous Sclerosis Alliance .

      References

        • Crino P.B.
        • Nathanson K.L.
        • Henske E.P.
        The tuberous sclerosis complex.
        New England Journal of Medicine. 2006; 355: 1345-1356
        • Curatolo P.
        • Bombardieri R.
        • Jozwiak S.
        Tuberous sclerosis.
        Lancet. 2008; 372: 657-668
        • van Slegtenhorst M.
        • de Hoogt R.
        • Herman H.
        • et al.
        Identification of the tuberous sclerosis gene TSC1 on chromosome 9q34.
        Science. 1997; 277: 805-808
        • Povey S.
        • Burley M.W.
        • Attwood J.
        • et al.
        Two loci for tuberous sclerosis: one on 9q34 and one on 16p13.
        Annual Human Genetics. 1994; 58: 107-127
        • The European Chromosome 16 Consortium
        Identification and characterization of the tuberous sclerosis gene on chromosome 16.
        Cell. 1993; 75: 1305-1315
        • Osborne J.P.
        • Fryer A.
        • Webb D.
        Epidemiology of tuberous sclerosis.
        Annals of the New York Academy of Science. 1991; 615: 125-127
        • O'Callaghan F.J.K.
        • Shiell A.W.
        • Osborne J.P.
        • Martyn C.N.
        Prevalence of tuberous sclerosis estimated by capture-recapture analysis.
        Lancet. 1998; 351: 1490-1499
        • Thiele E.A.
        • Jozwiak S.
        Natural history of tuberous sclerosis complex and overview of manifestations.
        in: Kwiatkowski D.J. Whittemore V.H. Thiele E.A. Tuberous Sclerosis Complex; genes, clinical features, and therapeutics. Wiley-Blackwell, Weinheim, Germany2010: 11-20
        • Krueger D.A.
        • Northrup H.
        • Roberds S.
        • et al.
        Tuberous sclerosis complex surveillance and management: recommendations of the 2012 international tuberous sclerosis complex consensus conference.
        Pediatric Neurology. 2013; 49: 255-265
        • Gillberg I.C.
        • Gillberg C.
        • Ahlsen G.
        Autistic behavior and attention deficits in tuberous sclerosis: a population-based study.
        Developmental Medicine & Child Neurology. 1994; 36: 50-56
        • Smalley S.L.
        • Burger F.
        • Smith M.
        Phenotypic variation of tuberous sclerosis in single extended kindred.
        Journal of Medical Genetics. 1994; 31: 761-765
        • Prather P.
        • de Vries P.J.
        Behavioral and cognitive aspects of tuberous sclerosis complex.
        Journal of Child Neurology. 2004; 19: 666-674
        • de Vries P.J.
        Neurodevelopmental, psychiatric and cognitive aspects of tuberous sclerosis complex.
        in: Kwiatkowski D.J. Whittemore V.H. Thiele E.A. Tuberous Sclerosis Complex; Genes, Clinical Features, and Therapeutics. Wiley-Blackwell, Weinheim, Germany2010: 229-267
        • Kopp C.M.C.
        • Muzykewicz D.A.
        • Staley B.A.
        • Thiele E.A.
        • Pulsifer M.B.
        Behavior problems in children with tuberous sclerosis complex and parental stress.
        Epilepsy & Behavior. 2008; 13: 505-510
        • de Vries P.J.
        • Hunt A.
        • Bolton P.F.
        The psychopathologies of children and adolescents with tuberous sclerosis complex (TSC). A postal survey to UK families.
        European Child and Adolescent Psychiatry. 2007; 16: 16-24
        • Hunt A.
        A comparison of the abilities, health and behaviour of 23 people with tuberous sclerosis at age 5 and as adults.
        Journal of Applied Research in Intellectual Disabilities. 1997; 1: 227-238
        • Staley B.A.
        • Montenegro M.
        • Major P.
        • et al.
        Self-injurious behavior and tuberous sclerosis complex: frequency and possible associations in a population of 257 patients.
        Epilepsy & Behavior. 2008; 13: 650-653
        • Eden K.E.
        • de Vries P.J.
        • Moss J.
        • Richards C.
        • Oliver C.
        Self-injury and aggression in tuberous sclerosis complex: cross-syndrome comparison and associated risk markers.
        Journal of Neurodevelopmental Disorders. 2014; 6: 10
        • Bolton P.F.
        • Park R.J.
        • Higgins J.N.P.
        • Griffiths P.D.
        • Pickles A.
        Neuro-epileptic determinants of autism spectrum disorders in tuberous sclerosis complex.
        Brain. 2002; 125: 1247-1255
        • Muzykewicz D.A.
        • Newberry P.
        • Danforth N.
        • Halpern E.F.
        • Thiele E.A.
        Psychiatric comorbid conditions in a clinic population of 241 patients with tuberous sclerosis complex.
        Epilepsy & Behavior. 2007; 11: 506-513
        • Lewis J.C.
        • Thomas H.V.
        • Murphy K.C.
        • Sampson J.R.
        Genotype and psychological phenotype in tuberous sclerosis.
        Journal of Medical Genetics. 2004; 41: 203-207
        • Raznahan A.
        • Joinson C.
        • O’Callaghan F.O.
        • Osborne J.P.
        • Bolton P.F.
        Psychopathology in tuberous sclerosis: and overview and findings in a population-based sample of adults with tuberous sclerosis.
        Journal of Intellectual Disability Research. 2006; 50: 561-569
        • de Vries P.J.
        Targeted treatments for cognitive and neurodevelopmental disorders and tuberous sclerosis complex.
        Neurotherapeutics. 2010; 7: 275-282
        • Joinson C.
        • O’Callaghan F.J.
        • Osborne J.P.
        • Martyn C.
        • Harris T.
        • Bolton P.
        Learning difficulties and epilepsy in an epidemiological sample of individuals with tuberous sclerosis complex.
        Psychological Medicine. 2003; 33: 335-344
        • de Vries P.J.
        • Prather P.
        The tuberous sclerosis complex (TSC).
        New England Journal of Medicine. 2007; 356: 92
        • Harrison J.E.
        • O’Callaghan F.J.
        • Hancock E.
        • Osborne J.
        • Bolton P.F.
        Cognitive deficits in normally intelligent patients with tuberous sclerosis.
        American Journal of Medical Genetics. 1999; 88: 642-646
      1. de Vries, PJ. (2002). The psychopathologies of attention on tuberous sclerosis. PhD Thesis, University of Cambridge, Cambridge, UK.

        • Ridler K.
        • Suckling J.
        • Higgins N.J.
        • et al.
        Neuroanatomical correlates of memory deficits in tuberous sclerosis complex.
        Cerebral Cortex. 2007; 17: 261-271
        • de Vries P.J.
        • Gardiner J.
        • Bolton P.F.
        Neuropsychological attention deficits in tuberous sclerosis complex (TSC).
        American Journal of Medical Genetics Part A. 2009; 149A: 387-395
        • Tierney K.M.
        • McCartney D.L.
        • Serfontein J.R.
        • de Vries P.J.
        Neuropsychological attention skills and related behaviours in adults with tuberous sclerosis complex.
        Behavior Genetics. 2011; 41: 437-444
        • Whittemore V.H.
        • Lewis J.
        Impact of TSC on the family and genetic counselling Issues.
        in: Kwiatkowski D.J. Whittemore V.H. Thiele E.A. Tuberous Sclerosis Complex: Genes, Clinical Features and Therapeutics. Wiley-Blackwell, Weinheim, Germany2010: 387-396
        • de Vries P.J.
        • Whittemore V.H.
        • Leclezio L.
        • et al.
        Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist.
        Pediatric Neurology. 2014; (in press)
        • Scriven M.
        Logic of evaluation.
        in: Mathison S. Encyclopedia of evaluation. Sage, Thousand Oaks, CA2005: 235-238
        • Goodman R.
        Psychometric properties of the strengths and difficulties questionnaire.
        Journal American Academic Child Adolescent Psychiatry. 2001; 40: 85-90
        • Berument S.K.
        • Rutter M.
        • Lord C.
        • Pickles A.
        • Bailey A.
        Autism screening questionnaire: diagnostic validity.
        British Journal of Psychiatry. 1999; 175: 444-451
        • Gioia G.A.
        • Isquith P.K.
        • Guy S.C.
        • Kenworthy L.
        Behavior rating inventory of executive function.
        Child Neuropsychology. 2000; 6: 235-238
        • Kushlick A.
        • Blunden R.
        • Cox G.R.
        A method of rating behaviour characteristics for use in large scale surveys of mental handicap.
        Psychological Medicine. 1973; 3: 446-478
        • Roach E.S.
        • Gomez M.R.
        • Northrup H.
        Tuberous sclerosis complex consensus conference: revised clinical diagnostic criteria.
        Journal Child Neurology. 1998; 12: 624-628
        • Northrup H.
        • Krueger D.A.
        • Roberds S.
        • et al.
        Tuberous Sclerosis Complex Diagnostic Criteria Update: Recommendations of the 2012 International Tuberous Sclerosis Complex Consensus Conference.
        Pediatric Neurology. 2013; 49: 243-354
        • Dancey C.
        • Reidy J.
        Statistics without maths for psychology: using SPSS for windows.
        Prentice-Hall, London, England2004
        • Kline P.
        The handbook of psychological testing.
        2nd Edition. Routledge, London, England1999
        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qualitative Health Research. 2005; 15: 1277-1288

      Linked Article

      • Tuberous Sclerosis Associated Neuropsychiatric Disorders (TAND) and the TAND Checklist
        Pediatric NeurologyVol. 52Issue 1
        • Preview
          Tuberous sclerosis complex is a multisystem genetic disorder with a range of physical manifestations that require evaluation, surveillance, and management. Individuals with tuberous sclerosis complex also have a range of behavioral, psychiatric, intellectual, academic, neuropsychologic, and psychosocial difficulties. These may represent the greatest burden of the disease. Around 90% of individuals with tuberous sclerosis complex will have some of these difficulties during their lifetime, yet only about 20% ever receive evaluation and treatment.
        • Full-Text
        • PDF