To develop a new instrument that assesses addiction severity among MMT patients we modified the MAP. Modifications to the MAP which included the addition of new items were made using: (1) an analysis of MAP scoring among opioid dependent patients from the Genetics of Opioid Addiction (GENOA) research collaborative, (14) consultation with experts in the field of addiction medicine (item-generation and assessment of the instrument’s face validity), and (2) a comprehensive literature review. To assess the psychometric properties of the new tool we preformed a generalizability (G) study. We lastly completed the criterion validation of the new addiction severity instrument using the GENOA sample of opioid dependent patients.
3.1. Genetics of Opioid Addiction (GENOA) Research Collaboration Methods
The GENOA research collaborative is a research initiative partnership between the Ontario addiction teatment center (OATC) and the population genomics program at McMaster University. GENOA is a prospective cohort investigation approved by the Hamilton integrated research ethics board. The GENOA research collaborative aims to evaluate the genetic determinants of methadone response. Participants were recruited from 10 clinical sites throughout southern Ontario, where they were then followed for 12-months. GENOA pilot study and methods have been described previously (15).
Information on medical history, methadone dose, duration on MMT, number of MMT treatments, and original source of opioid use was collected during the initial baseline assessment. The baseline assessment includes the M.I.N.I. International Neuropsychiatric Interview version 6.0 (16), which has been validated in the assessment of psychopathology. The baseline assessment also included instruments such as the MAP and brief pain inventory (BPI) to capture the severity and amount of interference pain has on a patient’s daily activities. Weekly urine drug screens were performed throughout the study period as part of routine clinical care at the OATC recruitment sites using the Innovacon, Inc One Step Drug Screen methods.
3.2. Item Generation
The important subdomains included in this tool were identified through collaborative discussions between psychiatrists, addiction medicine physicians, counselors experienced in MMT, methodologists and epidemiologists. These brainstorming sessions resulted in the identification of four subdomains: substance use, physical and psychological health, health-risk behavior as well as personal and social functioning. Using patient’s MAP scores from the GENOA investigation we determined items that most highly correspond with continued opioid use, as measured through urine toxicology screening. We also generated new items through an extensive literature review in addition to consultation with addiction medicine experts on the characteristics of “high-risk” methadone patients. Once we identified all relevant MAP/clinical/demographic items, we constructed our tool and distributed this tool to physicians working at the Ontario addiction treatment centers (OATC) in an effort to determine the face validity and acceptability of the instrument.
Exploratory factor analysis is often employed to identify domains during questionnaire development. Factor analysis is able to recognize variability among observed correlated variables. Within this study we modified an existing tool, thus utilizing domains previously identified by factor analysis and expert opinion. The extent of our modifications consisted largely of question addition and amendment. We felt it methodologically questionable to perform an exploratory factor analysis when utilizing questions from previously established domains.
3.3. Statistical Item Generation Results
We performed univariate analyses of patients’ responses to MAP items and opioid urine toxicology screening results. This allowed us to identify the most highly correlated MAP items for continued opioid abuse. The MAP scores among GENOA participants are summarized in Appendix B. Each MAP item was evaluated against participants’ opioid urine results using univariable linear regression. The results of the univariate analyses showed the number of days of illicit heroin, benzodiazepine, cocaine, crack and cannabis use as MAP items with high predictive ability for continued opioid abuse (P < 0.05, Appendix B). The results of the univariate analysis showed a higher severity of psychological symptoms are also associated with continued opioid abuse (P < 0.05, Appendix B). When determining the effect of risk-taking behavior on response to MMT, the number of days of injecting drug use in addition to a variable comprising the number of days of injecting drug use multiplied by the number of times per day correlate strongly with continued opioid abuse (P < 0.05, Appendix B). When exploring subdomains such as relationships (social/family), items calculating the frequency of conflict with friends proved significantly correlated with continued opioid abuse (P < 0.05, Appendix B). We tested additional clinical and demographic variables in an effort to generate additional items important for predicting response to MMT. Please refer to Appendix C for a summary of these results.
3.4. Expert Item Generation Results
Important developments arose when we consulted opioid addiction experts on the risk factors they encounter in their practice. The addiction medicine physicians suggested that we include an item that captures the environment patients live in, particularly whether the patient’s partner or close friends are still abusing substances, a factor that the most patients report upon relapse. The physicians also suggested we include items that capture work experience beyond the traditional “paid,” employment, with the greater significance being that we capture a patient’s responsibilities beyond paid work. This prompted us to also include “non-paid,” or “volunteer work,” as employment surrogates.
3.5. Literature Review: Item Generation Results
After carefully reviewing the literature, we identified additional items not captured by the MAP that may prove useful for predicting response to MMT. These items include: age of onset of opioid abuse, methadone dose, and duration on MMT. While not scored as part of the questionnaire, these items serve as important variables that should be captured when applying GREAT tool in a clinical setting.
Studies have demonstrated that longer duration of opioid abuse prior to and/or during treatment is associated with worsening health implications for at-risk populations. The risk of mortality from overdose, violence, infectious disease, and alcohol related causes are substantially higher among opioid addiction patients with long history of drug use (17, 18). One study also noted that patients who continue to use opioids into their 30’s face an unlikely chance of eventual opioid cessation (18). These results suggest that consideration of the age of onset of opioid abuse could prove helpful in the assessment of MMT patient response and prognosis.
Methadone dose is commonly discussed in the literature as a determinant for response to treatment. Although higher doses of methadone are associated with a higher risk of serious side effects, such higher doses are also associated with a positive treatment response (19).
Lastly, duration on OAT or “retention” in treatment is highly correlated with better health outcomes in opioid dependent patients (11, 20-23). The longer patients continued in a treatment program, maintain contact with a health care team and provide evidence of maintaining their will to remain abstinent, the better their treatment outcomes. Patients who leave treatment early or are just beginning MMT are vulnerable to relapse during their initial months often because of variable dosing, breakthrough withdrawal symptoms, and their vulnerable mental state (1).
3.6. Scale Generation
The MAP addresses questions through direct recall over a 30-day timeframe, where patients are asked to recall “how many times they injected drugs,” or “how many times they used heroin,” over a 30 day period. Directly asking patients to recall the number of times they have injected drugs or used a substance is subject to bias, for people’s ability to recall specific past events is often inconsistent (24). Memory recall is especially problematic for chronic conditions resulting in multiple recurrent events (25). Autobiographical memory is not organized by events such the amount of drinking or exercise one participates in, but rather by a hierarchy with prolonged periods at the top (i.e. during my first job I felt sad) (24). Thus, recall of the specific number of times people engage in a particular behavior should be regarded with doubt (24). To overcome such limitations we modified our assessment tool to include responses on a seven point Likert scale. Please refer to Appendix A to view the completed GREAT tool. The responses are ordinal and include seven categorical options. All questions were revised into a 7 point Likert scale format and piloted among addiction medicine experts. Further revisions addressed the language of questions, changing items such as “non-prescription,” to “not prescribed to you.” The experts helped to identify specific response scale descriptions that may limit the scale’s ability to capture variation between subjects. For example, the original response scale for the question: “In the last 3 months how often have you smoked cigarettes,” included categories such as (sometimes: 2 - 5 cigarettes per day, regularly 6 - 10 cigarettes per day, frequently: 1 - 11 pack per day). Addiction experts suggested that the categories would have limited the scale to capture variance between subjects since the majority (GENOA sample: 87%) of patients smoke more than a pack of cigarettes per day. Thus, our inclusion of so many categories below a pack of cigarettes (n = 20 cigarettes) would drive the majority of the participants to select 1 category (all of the time: > 1 pack per day).
The scale has been designed so that patients who score higher on the GREAT will be considered at a higher risk for continued opioid abuse and poorer MMT treatment response. All items have been categorized in an ordinal fashion with 7 being the “high risk” response. For example, we determined the amount of substance abuse by asking, “In the last 3 months, how often have you consumed opioids that were not prescribed to you?” Responses were scaled from 1 to 7, where 1 represented “not at all, 0 times,” and 7 represented “all of the time, every day.” For questions about educational achievement, education represented a surrogate component of socioeconomic status (SES). Lower SES is often equated with increased risk for poor outcomes, therefore the response category labeled 1 represented high educational achievement such as graduate university training, and 7 represented no formal educational training (elementary school or lower). It is suggested that all items on the tool be formatted similarly, reflecting the same scale categories (24).
3.7. Participant Testing
The tool was administered to opioid addiction patients from the GENOA study described above. A random sample of 21 GENOA participants from 4 clinical sites were administered the GREAT tool on 2 occasions with a 5-day interval between tests. The GREAT is intended to be a reliable and efficient self-administered tool for the assessment of a patient’s propensity for continued opioid abuse.
The 5-day interval was selected in an effort to ensure participants would not be influenced by their original answers while also maintaining the same level of addiction severity they had during the original test. Empirical data suggest that a re-test interval of 2 - 14 days is most appropriate for ensuring: 1) the attribute being measured has not significantly altered and 2) a minimal effect change on the magnitude of the reliability coefficient (24).
The criterion validation performed to assess the validity of the substance use domain against patients’ urine toxicology screens was performed on a sample of 117 MMT patients from the GENOA sample. This sample includes the 21 original participants used to test the reliability properties of the tool, using only results from their first test administration.
3.8. Statistical Analyses: Scale Modification, Reliability and Validity
To assess reliability we conducted a generalizability (G) study. G theory allows us to calculate reliability coefficients that acknowledge multiple sources of the errors of variance (24). Error is not only introduced when considering the between subject error, error can also be introduced with each test administration, and G theory allows for the integration of all possible sources of error into a single analysis of variance, and the individual variance components are computed (24). We performed a G study using the G-string software supplied by GENOVA (26). The reliability coefficients constructed from the G-study allowed us to determine the extent we can generalize 1) an individual’s scores across occasions (test-retest reliability), 2) items in one subscale to items in another subscale (internal consistency across subscales), 3) items within subscales (internal consistency within subscales) and 4) and individuals scores across subscales and test-administrations (average reliability).
When determining to what extent we can generalize across items, subscales and test administrations (average reliability), we set all facets of generalization to be random, and used the absolute error co-efficient in an effort to allow us to presume that each item, subscale or test administration is one of a random sample of all possible items, subscales or test-administrations (27). Using the relative error coefficients (Eρ2) would have restricted our ability to generalize results to just the items and parameters we have designed for this study (27).
We performed generalizability analyses for the overall test with all subscales included, providing reliability estimates for internal consistency and test-retest within each subscale. The design of our tool features crossed and nested facets. Our facets include time, subscale and item. Item is nested in subscale, however time and subscales are crossed facets. Our facets of generalization altered between reliability measurements (i.e. internal consistency, test-retest). Our facet of differentiation is the MMT patient, where we are measuring each patient’s addiction severity and level of response to methadone treatment.
There are currently no ways to objectively validate participant responses in the personal/social functioning or health risk behavior domains. These domains were created with the purpose of measuring high-risk behavior (e.g. sharing drug consumption paraphernalia), and social determinants of health such as intimate partner conflict, and criminal behavior. Items within these domains capture the behavioral and social context of participants, providing a more complete picture of how participants are functioning across different life aspects. This information is important to capture, however we are at a loss in identifying an objective method for criterion validation. Thus, we were limited to only performing a criterion validation on the GREAT substance use domain scores.
We performed a criterion validation using participant’s scores from the substance abuse domain of the GREAT using urine toxicology screening as the gold standard for assessing continued opioid abuse. As part of the GENOA protocol participants have provided up to date opioid urine screens, providing access to an objective measure of continued opioid abuse within the 3-month timeframe assessed in the GREAT. The urine data were used to determine whether or not participants had experienced an illicit opioid use in the three months preceding the questionnaire administration. We then used the participants’ 3-month opioid use history as the dependent variable in a logistic regression model. We used the participants’ GREAT scores from the substance abuse domain as the independent variable to determine the direction and strength of association between the GREAT substance domain responses (all answers summed to a maximum of 42 and minimum of 7) and 3-month history of opioid use. This model adjusted for age, sex, duration on MMT (years), and age of onset of opioid abuse, and methadone dose (mg/day). We also re-performed this analysis using three-month history of poly-substance use (THC, benzodiazepine, or cocaine) as the dependent variable. All statistical analysis were performed in STATA Version 13 (28).
LEAVE A COMMENT HERE: