Tag Archives: Duncan Watts

First Baby Step to Thinking of Evidence-Based Practice: Be Skeptical

At the recent 2012 conference of the International Society for AAC (ISAAC) there was some robust discussion about the technique know as facilitated communication. It’s a controversial technique and surprisingly one on which ISAAC does not have a position paper – which is an endeavor currently underway with a view to something being published soon. I say “surprisingly” because many other professional organizations have had position papers for many years, from the American Academy of Child and Adolescent Psychiatry (1993) through to the Victorian Advocacy League for Individuals with Disability [1]. ASHA has had a statement since 1994, so it does seem a little tardy for the group whose raison d’être is AAC to be publishing a statement on an AAC technique. But never mind, at least there is action being taken, which is better than continuing to say nothing.

But this isn’t about the pros and cons of FC. It’s about the development of a mindset that allows people to think about FC – and Non-Oral Motor Speech Exercises, Equine Therapy, Canine Therapy, Sensory Integration, and other such debatable practices. The reason I started with the reference to FC was simply because during the discussion, one person actually said, “But there’s more to this than Science.”

Is there? Is there really? I can appreciate that things in the world can be difficult to measure, and that there are times when measurement seems unfeasible and even intractable, but that doesn’t mean we stop trying.

Handbook of EBP in Communication DisordersEvidence-based practice can be tough. When you get into the nitty-gritty of the scientific method – which is a big chunk of what EBP is about – it’s easy to get overwhelmed by talk of variables, pre-tests, post-tests, levels of confidence, skewed distributions, ANOVA, one- versus two-tailed hypothesis, Bayesian, Cartesian, and the whole catastrophe that is experimental design. Even the most readable of books, such as the excellent The Handbook for Evidence-Based Practice in Communication Disorders by Christine Dollaghan [2], can be hard to read and even more challenging to digest. The potential complexity of designing ways to measure clinical practice is, to put it bluntly, off-putting. When you have a caseload of 200 clients and only 24 hours in a day, the idea of setting up formal measurement procedures is about as welcome as a bacon sandwich at a Bar Mitzvah.

Nil desperandum! Like any other skill in life, becoming a more effective practitioner of EBP doesn’t require you to be an expert all at once. You can improve your practice simply by sharpening your mindset to be more in tune with the concepts of EBP. And the first thing you can learn to do is become a Skeptic.

First, let me shovel out of the way that huge mound of steaming objection that being a skeptic is just an excuse for rejecting everything and believing in nothing. That’s a cynic, or a nihilist. In a 2010 interview with Skeptically Thinking, philosopher and author Massimo Pigliucci [3] said;

I think that a crucial aspect of being skeptical, of engaging in critical thinking, is not the idea that you reject claims because they seem absurd. That’s not being a skeptic, that’s just being a cynic. It’s just denying things for the sake of denying it. The idea of skepticism is that you inquire — that you do the work.

“Doing the work” is obviously a tough one because in our world of Wikipedia and endless cable shows about ghost hunters, psychics, celebrity hauntings, and quick-fix psychology, it’s easy to let someone else do the work for you – and that work may be of stunningly poor quality and accuracy. However, a little “critical thinking” is not that hard.

So here are my Top Three Critical Questions to help you become a baby Skeptic. And feel free to be skeptical about whether my three are a good three!

1. If someone claims X causes Y because they did Z, can the claim be tested independently? If I tell you that I can stop an interdental lisp by pushing the tip of a client’s tongue with a wooden spoon, while simultaneously saying “go back, tongue, go back,” you’d be right to ask if anyone else can do it, and you may even try it yourself. But if I claim that the reason no-one else can do it is because they don’t have the same spoon, or that my intonation pattern is very specific, you’d also be right to call bullshit on me.

2. If someone claims X causes Y because they did Z, are there any other simpler explanations as to why Y may have happened? When TV ghost hunters use a drop in temperature to “prove” the presence of a ghost, could something simpler have caused it? When a child appears to speak more after an hour with a dolphin, was it actually the dolphin’s presence causing it or just that the kids was happy?

3. If someone claims X causes Y because they did Z, what change was actually measured and how? “My kid talks more to my therapy dog, so therapy dogs work.” More than what? More than if there was a cat? More than 6 months ago? More than when he walked in the door? I had a client many years ago who swore blind that his stammer was much better after a few pints of beer and he wondered if he could get a prescription! Although I never took the opportunity to spend a night out at the bar with him, his measure of “better” was that he felt he was more fluent. But after a few pints of ale, I’m not sure my client was particularly accurate in his measurement techniques.

Everythiing is Obvious book

Oddly enough, I’m not going to suggest you use your common sense because this can be less “common” and “sensible” than you might believe. A recent book by Duncan Watts takes the notion of common sense to task. In Everything is Obvious: How Common Sense Fails Us, he argues that;

Common sense is “common” only to the extent that two people share sufficiently similar social and cultural experiences. Common sense, in other words, depends on what the sociologist Harry Collins calls collective tacit knowledge, meaning that it is encoded in the social norms, customs, and practices of the world.

Anyone who feels that common sense is in some sense the truth may want to spend at least 30 minutes listening to the discussions that go on in your country’s government, with folks in the US now facing 2 months of pre-election “common sense” being thrust down their throats. If sense were really that common, all parties in the political divides would cease to exist because their would only be one truth.

So common sense is less helpful in making evidence-based judgements than the basic science of testing and measuring. Even minimal measurement is better than no measurement because it gets you ever closer to an improved metric. You don’t have to subscribe to the “all or nothing” fallacy that some folks promote. Remember that there are different levels of measurement you can use, and each one has its pros and cons.

So let’s invent an example based on Dolphin Therapy. I can ask my client to tell me as much as possible about a picture of a busy street and record what is said, then repeat the task 5 minutes after spending a half-hour with a dolphin. If I simple count the number of words before and after the swim, then find the post-dolphin condition has twice as many words, is that a “good” measure? Well, the safest answers is “it’s a measure” but the notion of “goodness” is more complex. But here’s the valuable thing; you’ve at least created for yourself a methodology that you can use with the rest of your swimming clients. You can also do it again next time you client has another dolphin session. And the next.

Of course, don’t be surprised if someone else comes along and pokes holes in your methodology and results. The good news is you actually have some results to talk about, rather than a blanket statement about how “good for the kids” this dolphin fun is. Nor should you be surprised if someone uses the second question in my list to suggest an alternative explanation such as “the kid was just relaxed and would have done just as well if you’d given him a massage, or a bowl of ice-cream, or a flight in a helicopter.” This will help you go back and think of a better way to measure and test (or try to get a grant for “Helicopter Therapy” sponsored by folks who like flying in helicopters!) [3]

Enough for now. Once an article passes the 1500-word mark, it ceases to qualify as “baby steps.” So take those three critical questions and start trying them out. If you want some homework, try them out while watching a TV show about UFO’s or Bigfoot – it’s kinda fun.

[1] No, the “Victorian League” is not a group of steam-punk enthusiasts who yearn for a return to the values of the 19th century but an organization (VALID) based in the Australian state of Victoria, the capital of which is Melbourne.

[2] Dollaghan, C. A. (2007). A Handook of Evidence-Based Practice for Communication Disorders. Baltimore: Paul H. Brookes Publishing. This is great book and if you wanted to buy just one reference for EBP, I’d go for thisl But be warned; it is so full of excellent one-liners and summaries that if you use a yellow highlighter, there’s a fair chance you’ll end up with a banana-colored book. I use sticky tags and I think I went though three packs of them! And if you don’t want to spend the money – and time – on the book, you can read Christine’s 2004 ASHA Leader article entitled Evidence-Based Practice: Myths and Realities.

[3] Often the people promoting the benefits of animal therapy are animal lovers who appear to want to somehow “prove” that there’s something special about their dog/cat/dolphin/horse/lizard/three-toed sloth/whippet etc. I have no doubt that research shows how stroking a cat can reduce your blood pressure temporarily, but I can get the same effect from drinking beer, riding my motorcycle, or having sex. However, unlike the animal therapy folks, I am not promoting Drunken Biker Orgy therapy, or DBO as it would be referred to in the academic literature. Which may turn out to be a spectacular loss of revenue for me as a future project…

The Myth of the Digital Native

During the ASHA conference, I attended a really good presentation by a couple of excellent folks in the AAC field; Libby Rush and Celeste Helling. The topic was Making Evidence-based Decisions about Apps & SGD’s, which although a little weak in relation to the topic of actual metrics, did a great job of summarizing where the field is in relation to technology options and professional challenges.

One statement that Libby made is worth commenting on, particularly because it’s not uncommon. She said that one of her younger family members was able to use technology much more “intuitively” then she could and was more of a digital native. [1]

Baby at a laptop

“If I use a Boolean operator to invoke a recursive loop…”

Maybe it’s just me, but I find this notion that there is some special difference between digital natives, those who have grown up with computers and the Internet, and digital immigrants, those of us who have taken up using such technology as it was invented, just a little too simplistic and odd. The assumption is that any one-year-old can pick up an iPad and be surfing the net, sending e-mails, and twittering using the hash tag #imsosmartitscaresmyparents quicker than mom or dad can say “where’s the ON button?”

But a recent piece of research by the UK’s Open University found that the distinction between a native and an immigrant is slight to nothing. “We found no evidence for any discontinuity in technology use around the age of 30 as would be predicted by the … Digital Natives hypothesis,” said the report.  Rather than finding a correlation between age and effective use of technology, with “younger” being “more capable,” they found a stronger correlation between attitude to technology and techno-ability i.e. it doesn’t matter how old you are but how interested you are.

Given that the technology we are born into is always “normal,” the different between a native and an immigrant is a temporal thing, not an intellectual one. A laptop was not an item in my life until the age of 25 so it was, for me, “new technology.” For my daughters, they have always had a laptop in the house, and learned how to use them earlier than I.  In that sense, they are digital natives. But do they “know” more than I do about technology? No. Can they write better than I can? No. Do they understand the implications of posting personal information to Facebook imagining it to be private? No.  [2] And when the computer fails to boot (or gets infected by a virus), do they fix it themselves? Ha, that’s what Dad’s are for! [3]

Of course, you’re likely to argue that this is atypical, and that being an SLP working in Assistive Technology, I am a “special case” and that most people are not that tech savvy. But all this reflects is the difference between my attitude to technology (I am interested) versus my kids’ attitudes (they are ambivalent).  Adults with little interest in technology may well believe that because someone younger can (a) turn on a computer, (b) download a song from iTunes, and (c) spend hours on Facebook, they must be computer smart and way ahead of the curve. The truth is more prosaic – anyone who can do something you can’t do has the appearance of being smart. But if you were to spend a little time doing the same things, it wouldn’t then seem so magical. [4]

My wife, who always claims to be a computer idiot, actual runs our entire finances via the computer. She can track payments, shift balances, transfer money, pay bills, and all from a desk in our small home office. She can, in fact, do many things I cannot. Is this because she is a digital native? No, it’s because her attitude toward the tasks needing to be done is greater than mine; I can’t handle the accounts because I don’t need to!  Now, if we lose the connection to the Internet, she does not troubleshoot the network connection, nor will she reconfigure the software if there is a major problem. Why? Because I can do that and she is happy to let me. Does that mean she is “impaired” in her computer use? No, once again, it’s a matter of attitude and choices. Hey, my older daughter, who is a digital native, would similarly have no clue how to reconnect to the Internet – but she can spend hours on Facebook doing whatever it is she does. [5]

This helps to explain how we have a generation of technology users who can use Facebook, Twitter, smartphones, iPads, and the like in a seemingly effortless way, [6] but who have really little clue about how they work, what they do, what are the limitations, and what might be the dangers. Does anyone even worry just a tad when they post a message saying “I am at 555 West Avenue, Podunk, picking up my pay check?”

social nedia

The “digital native’s” digital life

None of this is intended to suggest that in order to use a piece of technology we have to know how it works. If someone backs a truck over my Droid and offers to hand me a free soldering iron and some duct tape to make amends, I’m afraid my response would simply be to make use of my monthly $6.99 insurance payment and get a new one. But there are two things that I think are critical to remember:

(a) Being able to use something does not make someone an expert, a whiz, a genius, a savant, or something particularly special.  That perception is nothing more than a reworking of the familiar lecturing trick “as long as I know 5% more than the folks I’m teaching, I’ll seem intelligent.”

(b) Your interest and attitude to technology is what makes you skilled at use, not whether you are a native or an immigrant – which is typically assumed to mean young or old.

So when I hear seasoned clinicians and educators suggest that they are somehow less savvy than “younger people,” perhaps you can understand why I want to give them a good shake and a group hug: If you’re presenting on AT to large groups of people and getting folks turning up regularly, believe me, you are a digital native!


[1] The distinction between digital natives and digital immigrants is purported to have been first used by the writer and educator Mark Prensky in an article entitled On The Horizon. The suggestion was that the Technological Era has changed how people think, and that “today’s students think and process information fundamentally differently from their predecessors.” At the time, this sounded radical and exciting but since then, critics have argued that it is simply not true, and that we all continue to process information in much the same way as we have done since losing our tails and cutting back on bananas. More strident critics have suggested that the digital natives are actually worse at processing information c.f. Mark Bauerlein (2008) The Dumbest Generation: How the Digital Age Stupifies Young Americans and Jeopardizes Our Future, and Nicholas Carr (2011) The Shallows.

[2] A startling number of people who use Facebook have a hard time understanding that the phrase “social network” means that information they post will end up on a “network” and that it is shared “socially.” I visited the Speech and Language Department of a nearby university where I found posters for sessions teaching students the dangers of such things as posting on Facebook because people such as future employers might read them. Might? You better believe that if you apply for a job for which I am on the interview panel I’ll be typing your name into Google “search” quicker than you can say “invasion of privacy!” The point is, if college students need to be taught the dangers of loose postings, then “digital natives” are not as smart as we think they are – or that they pretend to be.

[3] And after the first virus hit the home computer due to some dubious downloading practices by my dear daughters, it was also Dad’s job to remove the infection, beef up the firewall, install some new anti-virus stuff, and offer education on the dangers of file sharing – a practice digital natives are imbued with being able to do with ease but which turns out to be a dangerous affair because they don’t know the potential hazards!

[4] A word I like to use with regard to the affect (and effect) technology can have on people is enchantment. The near-magical way in which cool devices operate can suspend our disbelief and make us imagine that a piece of machinery can do wonderful, limitless things. Psychologically, enchantment can be dangerous as it discourages us from looking beyond the mask and subdues critical thinking. When someone believes that the answer to all the world’s problems can come from a piece of technology, then you are wise to be suspicious. The more sweeping the claims, the more skeptical you should become. Apparently, during the Industrial Revolution, machines were set to remove all need for hard labor and usher in lives of leisure. So how ‘s that working out for yah?

At some point, I’d like to do an entire post/article on Digital Enchantment: Breaking the Spell, but for now, I just have the title 😉

[5] OK, I’ll fess up that I know exactly what she does because I am on her “Friend” list and she forgets that whatever she posts is open to me and not just the person to whom she is posting. If need be, I can also log on as her because I know her password. So much for the super-smart digital natives! And before anyone tries to argue that I am a wicked father invading my daughter’s privacy, I have two comments: first, if it’s on Facebook it’s fair game; and second, as far as my family goes, I’ve always said that I don’t run a Democracy but a Benign Dictatorship 😉

[6] And there we go again with a nod toward another myth – that of the “intuitive device,” which leads to the dangerous notion that things should always be designed so that people don’t actually have to learn anything. Making something “easier” to use is a far cry from making it “intuitive.” In fact, there is no “intuitive” anything – it’s all learned. Check out Everything is Obvious: Once You Know The Answer (2011) by Duncan Watts for a sobering look at how we make assumptions about “ease of use” and “simplicity” yet forget what we’ve learned in the past. In a similar vein, Living With Complexity (2011) by Donald Norman talks about why things can’t always be simple, and how too much “simplicity” can be more complicated!