Friday, May 26, 2017

Risk Assessment Promise and Peril: The Colorado Experience

By David S. Prescott, LICSW


Background

The state of Colorado has long been at the forefront of attempts to develop effective methods for coming to terms with the risks posed by people who have sexually abused. In the days when each county or jurisdiction seemed to have different approaches, Colorado implemented the Containment Model. When there were no actuarial measures or other tools for structured professional judgment for grounding assessments, the Colorado Sex Offender Management Board (SOMB) assembled a list of 17 factors, which are a focus of the study below. For purposes of the study, these 17 items are treated as an alternative assessment measure, which was apparently not the original intention, although many evaluators have doubtless treated them as such. For its part, the SOMB is well aware that these 17 items are no longer the final word in assessments, even as they still receive consideration.

It seems important to note this background context, as Colorado’s efforts have indeed been pioneering over the years. In retrospect, it can seem easy to criticize the pioneering developments of groups of professionals. However, it should not be forgotten that when knowledge was scarce and approaches to sex crime resembled the Tower of Babel across the US, Colorado was among the first to develop approaches that numerous other states have emulated. Just the same, there is much we can learn from the study of these approaches, which is the subject of this blog.

The Research

An Online-First study by Katharine McCallum, Marcus Boccaccini, and Claire Bryson in the journal Criminal Justice and Behavior, offers fresh insight into the practical application of risk assessment research. The abstract describes their findings succinctly:

In Colorado, evaluators conducting sex offender risk assessments are required to assess 17 risk factors specified by the state’s Sex Offender Management Board (SOMB), in addition to scoring actuarial risk assessment instruments. This study examined the association between instrument scores, the 17 SOMB risk factors, and evaluator opinions concerning risk and need for containment in 302 Colorado cases. Evaluators’ ratings of risk indicated by noninstrument factors were often higher than their ratings of risk indicated by instrument results, but only their ratings of noninstrument factors were independently predictive of containment recommendations. Several of the most influential noninstrument factors (e.g., denial, treatment motivation) have been described by researchers as potentially misleading because they are not predictive of future offending. Findings highlight the need for more studies examining the validity of what risk assessment evaluators actually do, as opposed to what researchers think they should do.

This is not the first study finding that professionals often over-estimate risk across a range of conditions. The authors provide an eye-opening literature review, and Dr. Boccaccini has elsewhere found that the results of evaluations are often swayed according to who is paying for the service. For a context in which evaluators consider 17 items originally developed by the SOMB as a part of their evaluations over and above the far more scientifically proven actuarial measures, it is not surprising that evaluators would give extra weight to the SOMB measure and the items within it. In reading the study, several points become clear:

First, the evaluators in Colorado seem to face a difficult assignment, having historically assessed risk using items shown in research to have no predictive utility. What is the evidence-based assessor to do? Among the most heavily weighted items in the SOMB measure are defensiveness, psychopathology, and level of empathy, which are famously not associated with risk (and therefore with summary risk ratings), but are very likely strong responsivity factors to consider. This leads to questions as to what kinds of risk is actually being assessed, risk for sexual re-offense or risk for problematic adjustment to the conditions of community supervision. If it is the latter, perhaps the findings in this study might be more understandable – even appropriate – if the SOMB tool became more of a measure of risk, need, and responsivity? In this way, risk for sexual re-offense would be evaluated as a first hurdle, with treatment needs and the ability of the examinee to respond to treatment as the second and third hurdles of a more comprehensive assessment.  Whatever the case, this study suggests that many evaluators were not pursuing evidence-based approaches in making recommendations related to detention; this should be of concern to anyone interested in effective policy and human rights.

Adding to the complexity of the task, many of the SOMB items most considered in evaluations seem to overlap with items in actuarial scales such as Static-99r, the VRAG, and SORAG. Examples include criminal history, offense history and victim choice, and the nature of the person’s social support system. All of these lead to questions about conceptual double-dipping; how many times does one review criminal history before assessment results become skewed?

At the risk of appearing to be a Pollyanna, it is at least encouraging to see as much use of empirically validated measures as there is. It wasn’t that long ago that risk was assessed with little structure in the process and low accountability for the examiner (e.g., even including the physical attractiveness of the examinee). Although these findings point to much hard work ahead for professionals and policymakers alike, we can at least take heart that our methods have improved in many jurisdictions.

Just the same, the apparent conflation of responsivity and risk factors should cause any professional or lawmaker to be concerned. This comes along with the persistent overestimation of risk, and the means by which conclusions take shape. Further, as Boccaccini’s other research has shown, biases can enter the assessment process through any number of ways, whether explicitly or beyond the awareness of the examiner. This study reminds us that, for all of the rich scientific evidence at our disposal, we are still human beings, subject to being judgmental, opinionated, and biased.

Extending this last point further, one of the most interesting findings in this study was also one of the least explored. In the authors’ words: “In the current study, evaluator differences accounted for 8% of the variance in SOMB summarized risk ratings and 21% of the variance in summarized actuarial risk ratings” (p. 13). In other words, who the evaluator is can be a highly variable part of the equation. For all of our attempts to – and bluster about – the importance of impartiality, we have yet to reach the goal of remaining objective. In some cases, this may be an artifact of using relatively vague items. In other words, evaluator bias may even be akin to the famous country song: “Ya gotta dance with the one that brung ya.”

Final Implications

Reviewing both the study and the Colorado experience itself brought to mind a number of important reminders:

-          First, although these findings echo related findings elsewhere, it is still a single study
 
-         Second, it is always important to keep in mind that our best measures and best policies are always subject to bias at the hands of the individuals involved. Our ultimate work should be in the direction of professional self-development and consistency across groups of professionals.

Friday, May 19, 2017

Q & A with Christina Mancini entitled "Sexual Assault in the Ivory Tower: Public Opinion on University Accountability and Mandatory Reporting"

Mancini, C., Pickett, J. T., Call, C., McDougle, R. D., Brubaker, S. J., & Brownstein, H. H. (2017). Sexual Assault in the Ivory Tower: Public Opinion on University Accountability and Mandatory Reporting. Sexual Abuse: A Journal of Research and Treatment.


Highly publicized college sex crimes have recently captured public and policy attention. In response, greater discussion has turned to institutional accountability and controversial reforms such as mandatory reporting (MR). No study to date has measured public perceptions of campus sex assault procedures, however. This omission is notable because public opinion can directly and indirectly shape crime policy and because the topic has become increasingly politicized. Drawing on a 2015 poll of Virginia residents, this study evaluates views about campus sexual assault policy. Results indicate that two thirds of the public feel universities can effectively respond to sex crime and a large majority favors MR. Some differences in public opinion are evident. Research and policy implications are discussed.

Could you talk us through where the idea for the research came from?

The motivation of the study was to explore public attitudes toward a sensitive and increasingly politicized topic concerning sex crime, that is, perceptions of campus sex assault (CSA) policy in the U.S.  Our primary focus was on mandatory reporting and accountability views given growing public concern about how institutions of higher education (IHEs) are addressing and responding to CSA.  CSA affects everyone, not just those attending a college or university. IHEs are funded in part by public dollars and for that reason and others, the public would seem to be an important stakeholder in discussions and debates about how best to protect students and the campus community from sex crime.

We were fortunate to have access to a recent poll at the time, the Virginia Commonwealth University’s Commonwealth Education Poll conducted by the Wilder School of Government and Public Affairs in 2015.  The statewide survey included questions concerning resident support for MR and perceptions about CSA and accountability.  Virginia is one of the first states to adopt an MR policy and so it was a natural starting point for us.  Under the law, “responsible employees” at any public IHE (faculty, administrators) are required to report allegations that are disclosed to them to the university’s Title IX Coordinator.  From that point, the Coordinator, in consultation with a committee of the university community, determines if the allegation is serious enough to report to local law enforcement.  In such cases, victims are contacted by either the Title IX Office and/or law enforcement.  The law is controversial because some opponents view it as paternalistic as it reduces victim autonomy in reporting.  Others worry that is will divert valuable law enforcement resources for claims that are potentially falsified (e.g., say a student “discloses” sexual assault in a creative writing class assignment but the allegation is false) or unsubstantiated.

An overwhelming majority of the public, over 90 percent, in our study approved of Mandatory Reporting.  Factors associated with Mandatory Reporting support include believing in accountability, that institutions can reduce rape among students, and also, older age.  Less agreement surrounded the perception that university policy significantly reduces CSA.  Approximately two thirds of the public agreed that administrative action would reduce sexual assault among students.  The characteristics that influenced this perception of accountability included holding the belief that universities are generally unsafe places, higher education, and Democratic political ideology.

What kinds of challenges did you face throughout the process?

The greatest challenge was defending the scope and goal of the project.  All too often, there is a devaluing of public opinion research.  “Why care what the public thinks?”  “The public is ignorant.”  These statements illustrate the disdain I have run across in pursuing this strand of scholarship in academic circles.  Even colloquially, I have heard my students or others claim, “I don’t care what people think.”  But I would counter that as criminologists, we MUST care, we MUST measure, we MUST track perceptions, views, attitudes, and knowledge among the public.  Public perceptions, yes, are difficult to capture, but if we extended that logic to our subject matter, we would be out of business.  I would argue that public opinion research is no more elusive or less deserving of attention than tracking trends in sexual assault, evaluating the efficacy of treatment and intervention, or examining the criminal justice response to sexual offending.  Public views, for better or worse, matter in the creation of sex crime policy. 

Off the soap box for now, but the skepticism toward public opinion research was the major substantive challenge we ran across.  Our hope, to some small extent, is that we have made the case for why we MUST give serious empirical attention to a scholarly topic that is often ignored.

Of course, like other research, our focus could be further explored and refined.  We were challenged by some of the survey items which were general indicators of views about Mandatory Reporting and accountability, and also by the limited predictors of perceptions available in the poll.  The data were also from a single state and findings are not necessarily generalizable to national trends.  But challenges represent future opportunities, no?

What do you believe to be to be the main things that you have learnt about campus sexual assault?

The main contribution is that the public supports various efforts and measures to better respond to CSA.  Few divides in public attitudes toward Mandatory Reporting and accountability were evident.  Having said that, some members of the public held different perspectives concerning the law and administrative action.  These divides are deserving of additional examination.

Now that you’ve published the article, what are some implications for practitioners?

A policy lesson here is that the public appears concerned about CSA.  When do 90%+ of the public EVER agree on a controversial policy like Mandatory Reporting?  This large consensus, admittedly less so for accountability perceptions, suggests that administrators and policymakers consider efforts to engage the public in discussions and debates about CSA, particularly when it affects publicly-funded institutions. 

Christina Mancini, Ph.D.

Friday, May 12, 2017

Evidence-based practice for juveniles in 2017

 
For those of us who treat juveniles with sustained sexual offenses (JWSSO) a challenge is to identify the best methods. One common approach is to develop your own curriculum through trial and error, and good judgment, and find out what works for your clients and setting. These types of curricula are often "one-of-a-kind" developed locally, not using an existing model. I know numerous examples of these "Homebrewed" models that appear to be successful, and staff and client friendly.
 
This "Homebrewed" approach now is operating in an environment where an alternative, and possibly competing approach is Evidence-based Practice (EBP). The logic of the EBP approach is that resources are finite, and treatment models which are validated by appropriate research methods are the best choice. As I noted in an article (Ralph, 2012), this approach was delineated by Cochran and others. It was adopted by national organizations such as the Institute of Medicine, the American Psychological Association, and the Association for the Treatment of Sexual Abusers. Many county probation departments now require the use of EBP for programs they will fund.
 
A significant issue is what constitutes EBP? And can EBP be reconciled with the "Homebrewed" approach that appears to produce good outcomes? I noted (Ralph, 2012) there are several approaches to defining EBP. The traditional approach classifies treatment methods in terms of their research basis. The California Evidence-based Clearinghouse for Child Welfare (CEB4CW, 2012) developed one definition derived from the Institute of Medicine (2001), who defined EBP as: (1) best research evidence, (2) best clinical experience, and (3) consistent with patient values. CEB4CW's highest level of evidence for programs was described as, "Well-Supported by Research Evidence" which involved in part at least two randomized controlled trials published in peer-reviewed journals, documented in manualized form, and effective beyond a year. Only one program met those criteria for JWSSO, Multisystemic Therapy (MST) (Borduin, Schaffer, and Heiblum, 2009). Practically it is viewed as not a viable option for many counties because of the expense, extensive staff training, and supervision required. Also it was designed for high risk populations, where most JWSSO youth now are viewed as having relatively low risk (Caldwell, 2016).
 
Are there viable alternatives to the CEB4CW model and similar approaches? The most comprehensive research is regarding the general juvenile probation population, of which JWSSO is a subset. Dr. Lipsey at Vanderbilt University, and his colleagues, have carried out research using meta-analytic procedures (Lipsey, 2009). Their approach identified several factors connected with positive outcomes for programming for juveniles on probation. Notably they found positive outcomes, not only in "Namebrand" programs like MST, but also "Homebrew" or generic programs. Lipsey's group believes that an exclusive focus on methods with randomized trials, or other approaches using "Namebrand" programs, may leave out consideration of "Homebrewed" programs which may be of good quality, effective, and low cost. The importance of implementing programs with fidelity has been subsequently validated (Goense, Assink, Stams, Boendermaker, and Hoeve, 2016). Using Lipsey's research, and other studies, a list of program characteristics associated with positive outcomes, can be delineated. I will refer to this approach as Evidence-based Program Characteristics (EBPC) which can be described as follows:
 
 1. The risk level and needs of the target population is assessed using reliable measures.
 
2. A treatment approach addresses the risk level and needs of the target population, and  includes a sufficient amount of treatment to be effective.
 
3. The treatment approach uses social skill building, problem-solving, and counseling approaches.
4. The treatment method is manualized to reliably administer it.
 
5. Training and supervision is given regarding fidelity to the method.
 
6. Fidelity checks are "baked in" in and part of implementation of the method.
 
7. Reliable outcome pre/post measures are used to assess treatment effectiveness.
 
The opinion of the Lipsey group is these types of criteria are associated with a positive treatment effect.
 
While Lipsey's (2009) research is on the general juvenile probation population, what is evidence for JWSSO treatment? The best summary of existing research is by Reitzel and Carbonnel (2006). Every study in their research had a positive effect. While it was expected from previous studies that cognitive-behavioral programs would be more effective, also many locally developed "Homebrew" models also were. Kim, Benekos and Merlo (2015) studied the effect size of treatments for juvenile and adult sexual offending for reducing recidivism using meta-analysis of meta-analysis studies. They identified a medium effect size for adolescent programs, -.51.
 
Given the above findings, what are rational choices for the average county? The Reitzel and Carbonnel (2006), and Kim, Benekos and Merlo (2015) studies gives us cautious confidence that well designed studies, can show good outcomes. Likewise, if programs are implemented consistent with the criteria described above, can they be considered comparable to "Name brand" programs? I would tentatively assert this is correct. If these local "Homebrewed" programs also had conducted simple pre/post test studies showing effectiveness, and demonstrated low levels of recidivism compared to baselines, this would further strengthen the case for their appropriateness. Further, if they could also use experimental or quasi-experimental designs to assess outcomes, this would also help validate effectiveness.
 
This model for EBP, EBPC, is a reasonable approach for decision-making regarding JWSSO programs. While EBP may be seen as a competitor to "Homebrew" programs, these approaches may in fact be complementary if the approach described above is used to design those programs.
Norbert Ralph, Ph.D.
 
References
 
California Evidence-based Clearinghouse for Child Welfare. (2012). Scientific Rating Scale. Retrieved 6/1/12 from http://www.cebc4cw.org/ratings/scientific-rating-scale.
 
Borduin, C. M., Schaeffer, C. M., & Heiblum, N. (2009). A randomized clinical trial of multsystemic therapy with juvenile sexual offenders: Effects of youth social ecology and criminal activity. Journal of Consulting & Clinical Psychology, 77, 26 - 37.
 
 
Caldwell, M. F. (2016). Quantifying the Decline in Juvenile Sexual Recidivism Rates. Psychology, Public Policy, and Law. Advance online publication. http://dx.doi.org/10.1037/law000094
 
Goense, P. B., Assink, M., Stams, G. J., Boendermaker, L., & Hoeve, M. (2016). Making ‘what works’ work: A meta-analytic study of the effect of treatment integrity on outcomes of evidence-based interventions for juveniles with antisocial behavior. Aggression and Violent Behavior, 31, 106-115.
 
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
 
Kim, B., Benekos, P., & Merlo, A. (2015). Sex offender Recidivism revisited: Review of recent Meta-analyses on the effects of sex offender treatment. Trauma, Violence & Abuse, 17(1), 105–17.
 
Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4, 124-147.
 
Ralph, N. (2012). Evidence-based practice with juveniles. ATSA Forum, XXIV(3).
 
Reitzel, L.R., & Carbonell, J.L. (2006). The effectiveness of sexual offender treatment for juveniles as measured by recidivism: A meta-analysis. Sexual Abuse, 18, 401–421.
 
 
 

Friday, May 5, 2017

Cross border management of serious sexual and violent offenders

The ever increasing expansion of travel and the relocation of citizens for employment has resulted in vast movements of persons across the globe.  As people move so does crime, both organised crime penetrating across borders, but also the single transient offender seeking crime opportunities, the avoidance of regulations and management, and escape from stigma and surveillance at home.  A recent project completed for the European Union examined current issues and potential solutions for effective information exchange and management of serious sexual or violent offenders who move across EU borders.  Significant issues are presented by this group, not least because different EU states have differing risk assessment procedures, and differing categorisations of such offenders.  A number of Member States do not particularly identify such offenders, and in some Member States they are not subject to post sentences licences or sex offender registration.  This means in effect, that such persons are free to move across the EU without their Home Member State knowing where they are going, and can enter countries appearing as a ‘clean sheet’, with no notification to the receiving state about previous offences and current risks.  This has resulted in a number of tragic cases where EU nationals with significant previous convictions have re-offended seriously in the country they enter (e.g. Marek Harcar and the murder of Moira Jones in Scotland; or Robert M and the commission of 67 counts of rape of a minor in the Netherlands).

How can this situation be improved so that citizens are better protected? Firstly, it is possible to notify Member States about convictions. ECRIS (European Criminal Records Information System) can be used to notify a home state if a national commits a crime in another state (for example whilst on holiday or in employment).  This notification means the conviction can be known and appear on their record should they again come to notice at home.  For sexual offenders engaging in ‘sexual tourism’ this can be important to later risk assessments and court appearances.  Previous convictions can be requested on foreign nationals from their home state, again significant for accurate risk assessment, and critical for court appearances.  In terms of prevention, information can be exchanged via police personnel in Europol with designated officers passing information back to Member States, or by probation services.  This can be done even if a sentence has been completed, so long as the risk justifies it, and the exchange occurs in order to prevent crime.  A common approach to risk is required to facilitate such exchanges across a range of jurisdictions, and guidance with relevant templates can be found at: https://www.dora.dmu.ac.uk/handle/2086/12142   An overview of the information exchange mechanisms can also be found at: https://www.dora.dmu.ac.uk/handle/2086/12148

Data protection, confidentiality and privacy laws are often presented as barriers to effective information exchange in the European arena. However, there is a presumption for exchange if it is to prevent crime, and the potential harm and the vulnerability of victims (often children) can outweigh rights to privacy.  Practitioners should justify their decisions to disclose with reference to well evidenced risk assessments and that such a disclosure is necessary to the effective management of the person.  Further information can be found at:  https://www.dora.dmu.ac.uk/handle/2086/12143

Professor Hazel Kemshall (De Montfort University, UK) and Sarah Hilder, Ph.D., (Nottingham Trent University, UK).

For further information contact: kemshall@dmu.ac.uk

Additional references:

ECRIS (European Criminal Records Information System) http://ec.europa.eu/justice/criminal/european-e-justice/ecris/index_en.htm


EU (2017) Combating sexual abuse of children Directive 2011/93/EU. European Implementation Assessment. http://www.europarl.europa.eu/RegData/etudes/STUD/2017/598614/EPRS_STU(2017)598614_EN.pdf