Category Archives: Science

“I don’t care what the research says…”

A colleague of mine was asking for some references to support the notion that kids with severe learning difficulties can learn to use high frequency core words (such as want, stop, and get) because they were being told that what these kiddos really use (or need) are words like toy, cookie, and banana. I duly provided a quick sample of peer-reviewed articles and shared the information with other colleagues. And what the hell, I’ll share them with you, dear reader, in the References section at the end of this piece.

Reading the research

Reading the research

But another of my friends also commented that there are still those folks who respond with comment such as, “I don’t care what the research says, I don’t care who these kids are. These are not the kids I’m working with. The kids I’m working with just aren’t going to use these words.”

So what do you do about this? At what point does being “critical of the research” become “ignoring the research because I don’t believe it.”? In the world of Physics, it’s hard to say, “I don’t care what the research says, I’m still going to fly using my arms as wings.” Mathematicians don’t say, “I don’t care what the research says, 1 + 1 does equal 7.” And it’s a brave doctor who would say, “I don’t care what the research says, you go right ahead and smoke 40 cigarettes a day and you’ll be just fine.”

No-one would argue that Speech and Language Pathology as a profession will ever achieve the rigid, statistical certainties of physics and mathematics, but what does it say about our profession if we openly admit to ignoring “the research” because it doesn’t fit with our individual experience? There are certainly enough practices  in Speech Pathology that are hotly debated (non-speech oral motor exercises, facilitated communication, sensory integration therapy) and yet still being used. But all of these are open to criticism and lend themselves to experimental testing, whereas an opinion based on personal experience is not. I could tell you that I have used facilitated communication successfully, but that is still personal testimony until I can provide you with  some measurable, testable, and replicable evidence. This is one of the underlying notions of evidence-based practice in action.

However, it’s  one thing to talk about using evidence-based practice but another to actual walk the walk. If the evidence suggests that something you are doing is, at best, ineffective (at worst, damaging), how willing are you to change your mind? If 50% of research articles say what you’re doing is wrong, how convinced are you? What about 60%? Or 90%? At what level of evidence do you decide to say, “OK, I was wrong” and make a change?

If there’s anything certain about “certainty” it’s that it’s uncertain! Am I certain that teaching the word get to a child with severe cognitive impairments is, in some sense, more “correct” or “right” than teaching teddy? No, I am not. But what I can do is look at as many published studies of what words kids typically use, at what ages, and with what frequency, and then feel more confident that get is used statistically more often across studies. This doesn’t mean teddy is “wrong,” nor does it preclude someone publishing an article tomorrow that shows the word teddy being learned 10x faster than the word get among 300 3-year-olds with severe learning problems.

But until then, the current evidence based on the research already done is, in fact, all we have. Anything else is speculation and guesswork, and no more accurate than tossing a couple of dice or throwing a dart at a word board.

Being wrong isn’t the problem. Unwillingness to change in the face of evidence is.

References
Banajee, M., DiCarlo, C., & Buras Stricklin, S. (2003). Core Vocabulary Determination for Toddlers. Augmentative and Alternative Communication, 19(2), 67-73.

Dada, S., & Alant, E. (2009). The effect of aided language stimulation on vocabulary acquisition in children with little or no functional speech. Am J Speech Lang Pathol, 18(1), 50-64.

Fried-Oken, M., & More, L. (1992). An initial vocabulary for nonspeaking preschool children based on developmental and environmental language sources. Augmentative and Alternative Communication, 8(1), 41-56.

Marvin, C.A., Beukelman, D.R. and Bilyeu, D. (1994). Vocabulary use patterns in preschool children: effects of context and time sampling. Augmentative and Alternative Communication, 10, 224-236.

Raban, B. (1987). The spoken vocabulary of five-year old children. Reading, England: The Reading and Language Information Centre.

The Dudes Do ATIA 2013: Day 3 – Of Dining and Data

Today was a day of meetings. Fortunately, the first was at a delightful restaurant; the Thai Thani  on International Drive in Orlando. Being an Indian curry lover, I opted for the Curry Fried Rice with chicken, and wasn’t disappointed. One of the house specialities is a pineapple yellow fried rice curry with a choice of beef, chicken or pork, stir fried with raisins, cashews, and onions but I wanted something less fruity so I’ll save this special for another visit.

Thay Thani restaurant

Thai Thani Orlando

Following two more meetings, I did the first of my two joint-presentations. I usually fly solo – then there’s only me to blame of things go wrong – but this year I tried sharing. And this one was on one of my favorite topics: automated data collection and analysis with AAC devices. The content was similar to the presentation I gave at ASHA 2012 and which has already been documented in The Dudes Do ASHA 2012: Day 4, so feel free to click and read that.

What wasn’t discussed in that older post was the way on which the word data itself can tell us something about language change over time. So try this quick test – and don’t spend too long thinking about the answer:

Which is these statements is correct:

(a) The data is good.

(b) The data are good.

If you answered (b), then you are in the company of the good people at the  Oxford English Dictionary (and that’s not bad company to be in) and the hearts of die-hard grammatical prescriptivists [1].

But if you answered (a), then you are not that different from the population of the English-speaking world as a whole because the is and the are seem to be in free variation! If you take a look at the Corpus of Historical American English, you’ll see that in terms of frequency of use, they don’t seem to differ that much since the 1930′s, and you can make a case, I suppose, for arguing that the is-form has edged ahead of the are-form.

Take a look at these charts that track use since 1830.

The word data and the verb is

“The data is…”

Notice that “data is…” was being used at the turn of the century and peaked in the 1990′s. Compare that with the “data are…” instances:

The word Data and the word Are

The data are…

There are hardly any examples prior to the 1930′s and from the 1960′s onward, both is and are appear to be neck and neck in terms of usage.

So why does this happen? What is it that makes data such a tough word for folks to decide whether it should be used with is or are? The answer – or a t least part of it – is related to our understanding of whether a noun is a count noun or a mass noun.

For those saner readers who are less obsessed with language than this Dude, count nouns are – unsurprisingly! – those that can be counted. So dog, cat, shoe, table, boat, and cup, are all count nouns because we can talk about “three cups” or “five shoes” or “a room full of dogs.” With a count noun, you’re usually able to turn it into its plural form by adding an “s.”

On the other hand, a mass noun cannot be counted. Pork, education, furniture, and weather, cannot be used with a number or pluralized by adding an “s.” You don’t have “*three weathers” or “*a room full of furnitures.”

Data is one of those words that has become a mass noun, even though it was originally a count noun. And by “originally,” I mean going back to Latin, where the singular was datum and the plural was data. What often happens with foreign words that are imported into English is that we apply regular English rules to them. On that basis, it wouldn’t have been surprising to see datums – but it didn’t happen ;)

What appears to have happened is that the word data has become a synonym for information, and folks feel that if “the information is good” sounds OK, then so does “the data is good.”

Incidentally, there is a way to turn a countable noun into a mass noun by using a rather gruesome linguistic device called a “universal grinder [2].” Suppose that in a frantic effort to catch a bird that has found its way into your house, you cat leaps up into the air and accidentally hits a rapidly rotating heavy fan. Saddened by its untimely demise, you might, through your tragic sobs, explain to someone over the phone that, “There is cat all over the room.” In this situation, a regular count noun has suddenly transformed into a mass noun.

Kitten playing with a fan

Careful, Mr. Tibbles!

Equally, in certain circumstances, some mass nouns can take on the appearance of a count noun. Although water is typically a mass noun, you might be in a restaurant and remark  that, “there are four or five waters already on the table.” Needless to say, folks learning English have a bit of a struggle trying to learn the difference between them as the only rule seems to be that liquids and powders (amorphous items) tend to be mass nouns, and the rest are count.

The learning point from all this – and we’re trying to be recognized as an educational blog as well as providing entertainment – is that when we are evaluating someone’s ability to use language, it’s critical to be aware of the fact that sometimes the prescribed way of speaking may actually be in free variation with the popular way, and this is actually one of the ways in which language changes over time [3].

For the sake of completeness, the day ended with wine, pizza, beer (mass noun), and a cocktail before bed. Needless to say I fell asleep quickly.

Notes
[1] In the world of language mavens, there are constant arguments between prescriptivists, who take the line that there are “correct” ways to say things, and descriptivists, who say that so long as you can be understood, there ain’t no right and wrong.  Although I’m more often the prescriptivist boat, I’m happy to jump ship depending on my mood – and whether I want to just get into a bit of a row with someone just for the hell of it.

[2] The Universal Grinder is a linguistic thought experiment first written about by Francis Pelletier, who used it in a paper talking about the nature of count versus mass nouns. Pelletier didn’t use household pets and rotating blades as his examples but the Dudes feel more at home with Edgar Allan Poe as a role model than, say,  Noam Chomksy or Stephen Pinker.

Pelletier, F.  J. 1975. Non-Singular Reference: Some Preliminaries. Philosophia 5.

[3] A pretty comprehensive coverage of how and why languages change over time can be found in Larry Trask’s 2010 book Why Do Languages Change? For those who want the Dude notes, you can click on the following Dude Link to get the 38-page summary. Link to book summary

The Dudes Do ATIA 2013: Day 1 – Of Disclosures and Data

As if to prove that “the best-laid plans o’ mice an’ men gang aft agley,” one of the Dudes has had to cancel his trip to the ATIA conference here in Orlando, along with his presentation on eye gaze evaluation and implementation for children. Fortunately we’ll be able to make the notes available in a few days time, so check back or follow us on Twitter as @speechdudes.

The other Dudes – which would be me – is therefore left with the task of making sure folks get their dose of comments and observations inspired by the conference [1].

And these start with as a result of watching Jeff Smizek, the President and CEO of United Airlines, at the beginning of the flight to Orlando International. YOU may have to swipe your credit card to watch TV shows and movies on the tiny screen at the back of the chairs of a United Airlines Boeing 737-800, but Jeff Smizek’s little promo pieces are always free. In the latest, he enthuses over the new “control center” for the company based on one floor of the Willis Tower, former called the Sears Tower, in Chicago. From here, he extols, all of United’s operations are controlled and monitored in a state-of-the-art facility.

Boeing 737-800

Boeing 737-800

So Jeff, if something were to happen to the Willis Tower facility, wouldn’t that be a “bad thing?” And wasn’t the last time the words “Willis” and “Tower” seen together in the movie Die Hard when Bruce Willis battled against terrorists who took over the Nakatomi Tower? Do you really want to announce to anyone who flies United anywhere in the world the address of the entire company’s nerve center? Call me paranoid, but couldn’t the marketing department find something else to promote – like plane safety, passenger-centric improvements, or a short piece on how airline food is made? Just sayin’, Jeff.

During flights, I take the opportunity to catch up on reading materials that have been piling up over the weeks, which on this trip included the latest copy of Bloomberg Businessweek. What caught my eye was an article on the growth in the Dubai economy. Now I know you’re thinking –  “But Dude, what has this to do with Speech Pathology or Special Education?” Bear with me.

Consider the following paragraph taken from the article;

Rewa Zeinati, a freelance writer in Dubai, has noticed more business cards with photos of scantily clad women offering massages piling up on her car windscreen lately. “Sometimes I’m away for 30 minutes and come back to find a stack of them, ” says Zeinati. “I’ve definitely seen an increase this year.”

For those wanting to learn about experimental design, what we have here is a rather memorable example of the research dynamic duo of validity and reliability. You see, whenever you measure something (length, temperature, density, hair color, foot odor, number of toes etc., the two big questions you typically want answered are whether the test you apply actually measures what it is you are supposed to be measuring, and are these measures accurate.

Take, for example, the masochistic morning activity in which many of us indulge; checking our weight on the bathroom scales. Of late, the numbers I’ve been seeing having demonstrated – how shall I put it? – an “ascendant tendency.” My wife, on the other hand, would say, “You’re getting fat.”

My hope may be that there is a problem with the scales. Unfortunately, they seem to be reliable because they are consistent. By that I mean if I step off the scales and then back on, if they still show me at 170 lbs, they are consistent i.e. reliable. And if they appear to show my weight increasing slowly over time, and don’t suddenly drop to 130 lbs one day and up to 250 lbs on another, that’s more evidence of the reliability.

Bag of flour

Bag of flour

The next line of defense is to argue that the scales are mis-calibrated and are adding an extra 20 lbs to my “real” weight. They are, in fact, lacking in validity – they are not really measuring what they are supposed to be measuring. My wife then grabs two bags of flour from the kitchen, each weighing 5 lbs, and drops them on the scales. The first one shows up as 5 lbs, the second as 10 lbs. She then has me stand on the scales holding the flour and tragically the scales show 180 lbs. Validity confirmed, I walk off in a huff and sign up for a WeightWatchers class [2].

In the case of the Dubai economy, the first assumption of validity is that massage services are dependent on how much money is sloshing around in the economy, and that if folks have more disposable income, they spend more on executive relief. So if there are more cards appearing offering such services, this is an indicator that the economy is on the rise [3].  On the other hand, we might want to argue the reverse; that as an economy declines, people seek temporary relief from the misery of privation by seeking solace in the company of a masseuse, so more cards equals shrinking economy. Readers of the classic Hitch-hikers Guide to the Galaxy may recognize this as a variation on “Shoe Event Horizon” theory.

So it may be that the underlying assumption of the argument from the Businessweek article fails the validity test.

The other assumption is that the measuring rod – the number of cards on a windshield – is not accurate. Does each card measure X amount of “positive economy” or does the number of cards just fluctuate randomly over time? Without more longitudinal data – and looking at more cars than just Ms. Zeinati’s would also be critical.

So the card counting method may turn out to fail the validity test.

The Columbia Center for New Media Teaching and Learning has a simple, readable overview of validity and reliability as part of an on-line  tutorial about measurement in the social sciences, so it’s worth taking a look.

Signing in for the conference was a piece of cake. In fact, it was possibly the easiest sign in I’ve had for a while. A couple of days prior to arriving, ATIA sent me a confirmation email that included a bar code. All I needed to do was walk up to the desk at the conference hall, open the email on my smartphone, and wave the bar code at a sensor. My badge was automatically printed out and the only human element needed was for a person to put the paper with my name on it into a plastic holder. The day cannot be far off when even that job will be replace by having me fold my own paper and drop it into a holder I pluck out of a large box.

Registration barcode

Registration barcode

Dinner was at the Tropicale restaurant, one of the Caribe Royale’s on-site venues that I’ve eaten at many times, and I enjoyed a most excellent fire-roasted vegetable risotto served with chicken, asparagus tips, tomato fondue, and shaved Asiago. Delish! By the time dinner was over and beer had been consumed, my 5:00 am start ensured I was asleep within minutes of my head hitting the pillow.

Notes
[1] As we’ve mentioned before, our posts during conferences are not intended to be a comprehensive review of what’s happening but a series of reflections on what our conference experiences bring to mind. Rather than take 1000 words to outline the features and functions of an iPad app, we’re more likely to riff on the name of the app or mercilessly skewer one tiny aspect. If we say that H.L. Mencken is a role model, you’ll catch our drift.

[2] Astute – or anal – readers may want to raise the objection that we’re now making an assumption about the validity of using the bags of flour as a reference point. What if the makers of the flour have been unscrupulously shorting the contents by 0.2 lbs? Duly noted. It’s a fundamental issue in all measurement that whatever standards we use have to be valid – or assumed to be valid.

[3] I was tempted to use the phrase “…the economy and male anatomy are on the rise,” which is not only puerile play on the word rise but an example of something called syllepsis – a form of sentence where two or more parts of a sentence are yoked together by a common verb or noun, more often than not for humorous effect. Dorothy Parker allegedly once said, “It’s a small apartment. I’ve barely enough room to lay my hat and a few friends.” The sylleptic aspect comes from the dual meaning of the verb “to lay” and this type of humorous device is often called a pun – all be it a special version of a pun.

First Baby Step to Thinking of Evidence-Based Practice: Be Skeptical

At the recent 2012 conference of the International Society for AAC (ISAAC) there was some robust discussion about the technique know as facilitated communication. It’s a controversial technique and surprisingly one on which ISAAC does not have a position paper – which is an endeavor currently underway with a view to something being published soon. I say “surprisingly” because many other professional organizations have had position papers for many years, from the American Academy of Child and Adolescent Psychiatry (1993) through to the Victorian Advocacy League for Individuals with Disability [1]. ASHA has had a statement since 1994, so it does seem a little tardy for the group whose raison d’être is AAC to be publishing a statement on an AAC technique. But never mind, at least there is action being taken, which is better than continuing to say nothing.

But this isn’t about the pros and cons of FC. It’s about the development of a mindset that allows people to think about FC – and Non-Oral Motor Speech Exercises, Equine Therapy, Canine Therapy, Sensory Integration, and other such debatable practices. The reason I started with the reference to FC was simply because during the discussion, one person actually said, “But there’s more to this than Science.”

Is there? Is there really? I can appreciate that things in the world can be difficult to measure, and that there are times when measurement seems unfeasible and even intractable, but that doesn’t mean we stop trying.

Handbook of EBP in Communication DisordersEvidence-based practice can be tough. When you get into the nitty-gritty of the scientific method – which is a big chunk of what EBP is about - it’s easy to get overwhelmed by talk of variables, pre-tests, post-tests, levels of confidence, skewed distributions, ANOVA, one- versus two-tailed hypothesis, Bayesian, Cartesian, and the whole catastrophe that is experimental design. Even the most readable of books, such as the excellent The Handbook for Evidence-Based Practice in Communication Disorders by Christine Dollaghan [2], can be hard to read and even more challenging to digest. The potential complexity of designing ways to measure clinical practice is, to put it bluntly, off-putting. When you have a caseload of 200 clients and only 24 hours in a day, the idea of setting up formal measurement procedures is about as welcome as a bacon sandwich at a Bar Mitzvah.

Nil desperandum! Like any other skill in life, becoming a more effective practitioner of EBP doesn’t require you to be an expert all at once. You can improve your practice simply by sharpening your mindset to be more in tune with the concepts of EBP. And the first thing you can learn to do is become a Skeptic.

First, let me shovel out of the way that huge mound of steaming objection that being a skeptic is just an excuse for rejecting everything and believing in nothing. That’s a cynic, or a nihilist. In a 2010 interview with Skeptically Thinking, philosopher and author Massimo Pigliucci [3] said;

I think that a crucial aspect of being skeptical, of engaging in critical thinking, is not the idea that you reject claims because they seem absurd. That’s not being a skeptic, that’s just being a cynic. It’s just denying things for the sake of denying it. The idea of skepticism is that you inquire — that you do the work.

“Doing the work” is obviously a tough one because in our world of Wikipedia and endless cable shows about ghost hunters, psychics, celebrity hauntings, and quick-fix psychology, it’s easy to let someone else do the work for you – and that work may be of stunningly poor quality and accuracy. However, a little “critical thinking” is not that hard.

So here are my Top Three Critical Questions to help you become a baby Skeptic. And feel free to be skeptical about whether my three are a good three!

1. If someone claims X causes Y because they did Z, can the claim be tested independently? If I tell you that I can stop an interdental lisp by pushing the tip of a client’s tongue with a wooden spoon, while simultaneously saying “go back, tongue, go back,” you’d be right to ask if anyone else can do it, and you may even try it yourself. But if I claim that the reason no-one else can do it is because they don’t have the same spoon, or that my intonation pattern is very specific, you’d also be right to call bullshit on me.

2. If someone claims X causes Y because they did Z, are there any other simpler explanations as to why Y may have happened? When TV ghost hunters use a drop in temperature to “prove” the presence of a ghost, could something simpler have caused it? When a child appears to speak more after an hour with a dolphin, was it actually the dolphin’s presence causing it or just that the kids was happy?

3. If someone claims X causes Y because they did Z, what change was actually measured and how? “My kid talks more to my therapy dog, so therapy dogs work.” More than what? More than if there was a cat? More than 6 months ago? More than when he walked in the door? I had a client many years ago who swore blind that his stammer was much better after a few pints of beer and he wondered if he could get a prescription! Although I never took the opportunity to spend a night out at the bar with him, his measure of “better” was that he felt he was more fluent. But after a few pints of ale, I’m not sure my client was particularly accurate in his measurement techniques.

Everythiing is Obvious book

Oddly enough, I’m not going to suggest you use your common sense because this can be less “common” and “sensible” than you might believe. A recent book by Duncan Watts takes the notion of common sense to task. In Everything is Obvious: How Common Sense Fails Us, he argues that;

Common sense is “common” only to the extent that two people share sufficiently similar social and cultural experiences. Common sense, in other words, depends on what the sociologist Harry Collins calls collective tacit knowledge, meaning that it is encoded in the social norms, customs, and practices of the world.

Anyone who feels that common sense is in some sense the truth may want to spend at least 30 minutes listening to the discussions that go on in your country’s government, with folks in the US now facing 2 months of pre-election “common sense” being thrust down their throats. If sense were really that common, all parties in the political divides would cease to exist because their would only be one truth.

So common sense is less helpful in making evidence-based judgements than the basic science of testing and measuring. Even minimal measurement is better than no measurement because it gets you ever closer to an improved metric. You don’t have to subscribe to the “all or nothing” fallacy that some folks promote. Remember that there are different levels of measurement you can use, and each one has its pros and cons.

So let’s invent an example based on Dolphin Therapy. I can ask my client to tell me as much as possible about a picture of a busy street and record what is said, then repeat the task 5 minutes after spending a half-hour with a dolphin. If I simple count the number of words before and after the swim, then find the post-dolphin condition has twice as many words, is that a “good” measure? Well, the safest answers is “it’s a measure” but the notion of “goodness” is more complex. But here’s the valuable thing; you’ve at least created for yourself a methodology that you can use with the rest of your swimming clients. You can also do it again next time you client has another dolphin session. And the next.

Of course, don’t be surprised if someone else comes along and pokes holes in your methodology and results. The good news is you actually have some results to talk about, rather than a blanket statement about how “good for the kids” this dolphin fun is. Nor should you be surprised if someone uses the second question in my list to suggest an alternative explanation such as “the kid was just relaxed and would have done just as well if you’d given him a massage, or a bowl of ice-cream, or a flight in a helicopter.” This will help you go back and think of a better way to measure and test (or try to get a grant for “Helicopter Therapy” sponsored by folks who like flying in helicopters!) [3]

Enough for now. Once an article passes the 1500-word mark, it ceases to qualify as “baby steps.” So take those three critical questions and start trying them out. If you want some homework, try them out while watching a TV show about UFO’s or Bigfoot – it’s kinda fun.

Notes
[1] No, the “Victorian League” is not a group of steam-punk enthusiasts who yearn for a return to the values of the 19th century but an organization (VALID) based in the Australian state of Victoria, the capital of which is Melbourne.

[2] Dollaghan, C. A. (2007). A Handook of Evidence-Based Practice for Communication Disorders. Baltimore: Paul H. Brookes Publishing. This is great book and if you wanted to buy just one reference for EBP, I’d go for thisl But be warned; it is so full of excellent one-liners and summaries that if you use a yellow highlighter, there’s a fair chance you’ll end up with a banana-colored book. I use sticky tags and I think I went though three packs of them! And if you don’t want to spend the money – and time – on the book, you can read Christine’s 2004 ASHA Leader article entitled Evidence-Based Practice: Myths and Realities.

[3] Often the people promoting the benefits of animal therapy are animal lovers who appear to want to somehow “prove” that there’s something special about their dog/cat/dolphin/horse/lizard/three-toed sloth/whippet etc. I have no doubt that research shows how stroking a cat can reduce your blood pressure temporarily, but I can get the same effect from drinking beer, riding my motorcycle, or having sex. However, unlike the animal therapy folks, I am not promoting Drunken Biker Orgy therapy, or DBO as it would be referred to in the academic literature. Which may turn out to be a spectacular loss of revenue for me as a future project…