24.3 Why Most Modern Research is Flawed with Ken Gillman, MD

[00:00:00] I see so many patients who come to me with depression that have had five, six, seven trials of SSRI, SNRI. And I just say, look, I don't know for sure what's going on with you or, you know, I can't predict exactly what maybe the underlying diagnosis is. But I'll tell you one thing. We're going to do something different than the other seven times you've been treated in the past. Somebody get this guy some help.

[00:00:36] Disclaimer. This podcast is for informational purposes only. The information provided in this podcast and related materials are meant only to educate. This information is not intended as a substitute for professional medical advice. While I am a medical doctor and many of my guests have extensive medical training and experience, nothing stated in this podcast nor materials related to this podcast, including recommended websites, texts, graphics, images, or any other materials, should be treated as a substitute for professional, medical, or psychological advice, diagnosis, or treatment. All listeners should consult with a medical professional, licensed mental health provider, or other health care provider if seeking medical advice, diagnosis, or treatment. Or, put more simply,

[00:01:04] If you need help like this guy, call your own doctor. And that's one thing that's never been done with antidepressant drug trials. I know it's a little bit more difficult because everything's a little bit less definite and certain and all the rest of it. But if people had spent more time looking carefully at which symptoms to assess and how to assess them, if there's a relationship between the administration of an antidepressant drug and the improvement of symptoms, and you plot the symptom changes in large numbers of patients,

[00:01:32] you would expect to clearly see that the maximum improvement in people who are going to be long-term responders occurs within a discrete time frame of two to four weeks. Now, of course, many people, like people who are experienced in using MAOIs, because they produce such a definite and rapid effect in a substantial proportion of patients with melancholic depression, you see definite changes within five to ten days in a substantial proportion of people.

[00:02:03] And I think many people in the group who, sorry, when I say the group, I mean the international MAOI expert group that I founded several years ago, would agree that that's the case, that if people don't get any response in that time frame, they're probably not going to get a response later on. Obviously, you've got the variant, but there must be some people who need a rather larger dose. So, although whatever it is, three or four might be enough for the average person. Some people need five or six tablets, 50, 60 milligrams.

[00:02:32] Well, in the case of Parnate, Chalcypramine. But leaving that aside, you obviously expect a response within a discrete time frame. And that would help you establish the cause-effect relationship. That's where Pearl's work and a more finely dissected analysis of how people respond to drugs like tricyclic antidepressants might well help us to decide which properties of those drugs

[00:03:00] might actually be responsible for the improvement of which groups of symptoms. I mean, like, for instance, we know for sure that the sedative appetite effects are mediated by H2 receptors. H2 receptors, isn't it? I sometimes get my receptors muddled up. And if you block H2 receptors with an antihistamine, people feel sleepy within a few hours. And it lasts until the drug gets out of the system

[00:03:29] and then they feel less sleepy, right? Whereas antidepressant effects take five to ten days or whatever. So you've immediately got a mechanism there by which if you look at the cause-effect relationships and the mechanism, you can dissect out what's happening and which bits of the syndrome are improving as a result of H2 blockade as opposed to inhibiting the serotonin pump or the noradrenaline pump or whatever. Yep. And almost none of that kind of work's been done

[00:03:58] in the last 50 years. I did a review of this. I don't claim that I would have identified every single relevant paper, but I think it was fingers of less than one hand counted the number of studies that had actually looked at that. I think that's extraordinary. Actually looked at, you mean the time response? Yeah. And whether it's different with different drugs, different types of symptoms, anxiety versus anhedonia versus blah, blah, blah.

[00:04:27] So would you say that is beyond the greater reduction in symptoms over the long term? Would you say that is in one of the kind of basic science reasons why you feel like MAOIs work so much better than SRIs because there is a predictive response in a predictive amount of time? Yes. I mean, that's undeniable, isn't it? But back to Pearl, you know, you're not doing science

[00:04:55] if you're not looking at causes and mechanisms. And you have a natural course of melancholic or biologic depression that will tend to subside in a given timeframe. And so if you have an unpredictable time response to your drug treatment, then you start to get into the likelihood that the person may have just been coming out of that depressive episode naturally. But then if you're, you know, the company paying for the trial,

[00:05:24] you want to say, no, it was the drug that I gave that it just took this one eight weeks to work. Yes, that's right. You can't have your cake and eat it too. Obviously the fly in the ointment is to some extent the variability in the time response of the natural course of the illness. Although that's perhaps not quite such a big problem in the sense that if your response to treatment is less than two weeks and the usual course of the illness is months rather than weeks,

[00:05:52] then that's not quite such a confounding factor as you might at first expect. And also, of course, it would be susceptible to the type of investigation with large numbers of patients in terms of plotting time course. If it's natural remission, then that should be unequally distributed along the time frame. Whereas if it's response to treatment, it should occur within a discrete time window. So you're looking at the differences

[00:06:19] in variability across the response over time. For example, just to bring it back to what we're really talking about with MAOIs versus TCAs versus SRIs. And so with MAOIs, and I don't know, like you said, I don't know of any specific studies that look at this, but I would guess if you looked at the MAOI response in melancholic depression, you would see a much tighter variability of how long it takes

[00:06:49] those patients to respond. Whereas with SRIs, if you have a response, it could be on day one or it could be eight weeks later. I think the variability with that response is going to be much more variable. But again, going back to the whole question of clinical practice and clinical judgment, I mean, especially in the days of TCAs, I would frequently have patients who would perhaps be referred to me taking amitriptyline

[00:07:17] because the primary care doctor might give them that and I would swap them to clomipramine. They'd get a major symptom improvement with clomipramine. Clomipramine is undoubtedly the most potent of all the tricyclic antidepressants, despite the fact that in North America for a long time, it was only ever used as an anti-obsessional drug. But there you are. And let's say, as part of practical clinical observation and experiment, after people had been on it for a month, they'd come back and say, look, this is much, much better than amitriptyline doctor,

[00:07:46] but the side effects are much worse. I can't reach a sexual orgasm and blah, blah, blah. Can I go back on the amitriptyline? I'd say, yeah, sure. Okay. I think you'll find you worse on the amitriptyline or I might not even say that. But yes, if you think you'd like to try going back on, they'd go back on the amitriptyline and they get worse again. So perhaps I'll go back on the clomipramine. Maybe I can get better with a slightly lower dose and not have so many signs. Yeah, okay. And they get better. So you're doing an experiment, a series of experiments where one drug doesn't work, you swap it,

[00:08:15] you swap back again. And then with the dose, you know, you get people up to 150 in clomipramine. They're a hell of a lot better, but they've got side effects which are slightly troublesome. So you say, okay, look, we've done a blood level. It's slightly high. Let's just back off on the dose a little bit. You know, take 25 milligrams less for a month, see what happens. Okay. Do the same thing next month. Then you get down to 75 milligrams and they get worse again. So you increase it back to 100. They get better again. Well, that's Judea Pearl's do operator.

[00:08:44] You're changing the input, i.e. the dose, and you're repeatedly proving that the higher dose works and the lower dose doesn't. Well, that's inherently better science than a randomized controlled trial. That's a really important thing to recognize. So well done clinical practice where people are making proper observations, recording them properly, doing it systematically, where you do that kind of thing. That is better science

[00:09:13] than a randomized controlled trial. So what really upsets me is it seems to me that there is a whole huge group of doctors who seem to have been bullied into accepting the idea that they have to do what a randomized controlled trial says and their individual experience of individual patients is of lesser value or no value at all. And I think that's the biggest mistake in clinical science that has been made

[00:09:41] in the last 50 years. And the joke is people aren't learning from history. If you look at that article I've written about RCTs, I quote a whole shed load of people who have said this. Let me see if I can find it. Slow motion train wreck? No, not that one. Actually, this one may not be up on the website yet. I must pull my finger out and do it. Lewis Lasagna was the head of the FDA in the early 60s

[00:10:10] when DESE drug experiments that was the study where they wanted all the drug companies to provide new evidence of the effectiveness of drugs in order to introduce a more scientific assessment of them and that was when randomized controlled trials gained their hegemony. Hegemony, for those people who prefer to say it like that. Lasagna, in the mid-60s, said the pendulum

[00:10:38] had swung too far already in the 1960s. In contrast to my role in the FDA, he means, in the 1950s, which was trying to convince people to do controlled trials, now I find myself telling people that that is not the only way to the truth. Most knowledge comes from naturalistic observations and smart physicians using their past knowledge

[00:11:07] and experience as control. That was the fellow in charge of the FDA in the 1950s and one of the, Paul Lieber, who was head in the 80s, said something similar as well. Sir Michael Rawlins, who was head of the NHRA, whatever they call that in England, of medical research and health organisation or something, the notion that evidence can be reliably placed in hierarchies

[00:11:36] as all guidelines do is illusory. RCTs have important limitations of which four are particularly troublesome. Not just four, there were more than that, but he thought these four were particularly troublesome and so on and so on and so on. So there were shed loads of people who were involved in clinical medicine at the very highest level who've repeatedly said over the years that there's too much

[00:12:05] emphasis on randomised controlled trials. As a younger practitioner, you know, in the world that we live in, everything is so fast-paced and practising like that clinically takes time. I tell my patients, I don't know that I'll be able to get you better in the next month. But what I can promise you is that I will be methodical about how I go about things. Changing one variable at a time, understanding the influence of

[00:12:35] outside variables, but I do wonder if the newer generation, because I feel like this a lot of the time with, you know, these things buzzing a thousand times a day and emails, the internet, the massive expanse of information, it gets all of us so sped up and I think when we're sitting in front of a patient, well, we've had the training, we've passed all the tests, we're supposed to have the knowledge of exactly what to do and help that patient

[00:13:05] bring them out of that depressed state or anxious state or obsessional state as rapidly as possible. But I think in reality, the wiser thing is to let your patient know this may take a little bit of time, but we will figure out what treatment is the best for you. and you use the randomized control trial, you use the evidence out there to try to better understand, but you keep an open mind about what will work best in your patient population.

[00:13:35] And I think that's where this business of randomized controlled trials and guidelines has been demonstrated another very negative aspect and that is the number of doctors now who won't or are reluctant or anxious about using treatments which aren't in the guidelines. And the irony about this is that if you look at the intro or the small print for all

[00:14:04] the well-produced guidelines by reputable bodies and not all of them are reputable, I mean, there's one paper I found a while ago which was a guideline to using guidelines. I mean, what a joke that is. I got to look that one up. I'm sure I can find it quickly for you. They're frightened of treating people as individuals. So guidelines say something about groups, but extrapolating

[00:14:33] what guidelines say about groups, it's the external validity problem. I remember seeing two papers relevant to this. One is asthma. I have a mild asthma. If you look at the asthma trials, only about 5% was it or something of patients being treated for asthma by doctors actually fulfill the criteria to enter the trials for anti-asper drugs. Well, that just shows you that the extrapolatability and the external validity

[00:15:03] of them is next to zero. This same thing came up with rise to depression, of course. If you look at, I think, Zimmerman, right? Perhaps. There's more people picking up a couple of references to this when I was writing about it a while ago. For depression, it's something like 75% of the patients being treated for depression would not be 75% on the low end of the confidence interval. On the high end, it was 99.1%. Okay, right. Yep. Okay. So send me that reference.

[00:15:33] I might add it to my collection. But, I mean, that alone tells you something really important, doesn't it? So, but, so anyway, back to what I would like to be a really powerful message to practicing clinicians, and that is believe in yourself and believe in your clinical judgment and build up your practical clinical experience doing proper science of cause-effect relationships, altering the supposed causal factor, whether it's the dose of a drug or whatever else it

[00:16:02] is, and carefully documenting and studying the results. And that will give you far more valuable knowledge than randomized control trials ever can or ever will. And for God's sake, or the flying spaghetti monster's sake, if your patient comes to you and they have failed multiple trials of an SSRI, please don't keep giving them alternative SSRIs and

[00:16:32] thinking that they're going to work. That, that, I cannot tell you, I see so many patients who come to me with depression that have had five, six, seven trials of SSRI, SNRI, and I just say, look, I don't know for sure what's going on with you or, you know, I can't predict exactly what maybe the underlying diagnosis is, but I'll tell you one thing, we're going to do something different than the other seven times you've been treated in the past.

[00:17:02] I couldn't agree with you more. It's appalling, isn't it? And look, I guess I probably have a fairly unusual spread of experience in this regard, Ethan, because of course I get consulted by people all over the world. And I can tell you that people from all over the world say exactly the same. I've had five different SSRIs and three different supposed SNRIs. I think you know, I think half the SNRIs

[00:17:31] aren't really SNRIs at all anyway. Well, then the vaccine, right? So much norepinephrine with vinlafaxine. Yep, exactly. Yeah, it's sarcasm. Yeah, that's a really important thing to recognize. Yeah. Yeah, it's wild. Just briefly a little bit more, if you could just give us a kind of a brief history, because I do try to kind of promote these messages to not just young practitioners, but also curious patients. can you run through kind of the history of

[00:18:01] how depression was conceptualized and treated in the 1950s and how that evolved to where pretty much every diagnosis in the DSM-5, the guidelines say to use an SSRI? No. I don't think I can do that off the top of my head. That was a fairly general question. Can you make the question a little bit more specific?

[00:18:30] How did we get from using an effective treatment class like MAOIs to 50 years later using a much less effective treatment if we're talking about melancholic or biologic depression? To be sure, to be sure, you think I can tell you that. This, I have Irish ancestry. I think this relates to the splendid joke

[00:19:00] that goes a long way back about Paddy and Seamus coming out of the pub one night and splendid evening they've had. As they're coming out of the pub, Paddy bumps into a friend of his and Seamus goes ahead to get the car. When Paddy catches him up, he finds him on his hands and knees in the gravel on the side of the road. He says, what are you doing there Seamus? I'm looking for the car keys, Paddy. I dropped them. It fell out of my hand. I just dropped them. I did. And Paddy

[00:19:30] says, where did you drop them? And he said, oh, you dropped them over there, Paddy. What are you looking here for then, Seamus? He said, well, this is where the night is. Now that tells you the answer to your question, right? Because, of course, we think that the drug we've currently got is the one that works for all sorts of reasons related to what we've been discussing for a while, right? So when SSRIs appeared,

[00:20:00] they were the bees and knees, they were advertised, you know, and the money that drug companies have into influencing the whole system through adverts and God knows what else is overwhelmingly powerful. And so irrespective of what teaching and science might have said, SSRIs became, you know, the flavor of the month. And, well, decade rather, I should say, whatever. Yeah. Century. One of the most delightful professors I ever worked with in doing a

[00:20:29] little research was Professor Curzon, George Curzon, who was the professor of neurochemistry at Queen Square in London. I remember sitting with him in his office one day and he was the king of 5-HT research. And, you know, we were analyzing tissues to measure their 5-HT levels. And I can't remember exactly what the conversation was, well, why do we concentrate on 5-HT so much? And George said, well, the truth is,

[00:20:59] Ken, that we're not really able to measure anything else. And that's, absolutely true. I mean, you know, back in, well, people didn't know about trace amines and they couldn't measure them. So nobody looks for something they don't know about and can't measure. You also had the citra, you had mentioned a while back about the causality of cigarettes and lung cancer and how, you know, that message was not promoted to the general public

[00:21:29] because there was a lot of vested interest in big tobacco. And then I don't remember, Ken, was it you that talked about how some of the biggest names or the biggest investors in big tobacco, when they couldn't stand behind that varsity that smoking did not lead to lung cancer or wasn't a big causative factor, that they skipped out on the big tobacco market and they went over to big food and then they started putting all kinds of

[00:21:58] crappy additives that made us addicted to the food that we eat. Merchants of doubt. Naomi Oreskes, she does a column in Scientific American or somewhere, I've forgotten what she's professor of and where, but she's very good. Naomi Oreskes, that covers all of that kind of thing in great detail. I would recommend, I mean, look, there are endless drugs written by Healy and Professor Shorter and

[00:22:28] heaven knows who else about the history of Prozac, the Prozac era and all that kind of thing. This book is a little bit more general and gives you a slightly wider and bigger overview of the whole business of how vested interests affect the way people think, how they think and what they think they think. I wonder if Dr. Healy knows Paddy and Seamus. Oh, I'm sure he does, yes. So, folks, don't forget

[00:22:58] Seamus. I'm looking here because that's where the light is. And he's still there, he's still looking. Yep, he's still looking. And so are we. No, that's a really important thing to keep in your mind. Why are we doing this? And sometimes the answer is, well, because it's the only thing we know how to do. Yeah. Anything else that you want to add or to wrap up our conversation that we didn't get a chance to talk about?

[00:23:28] I don't think so, no. I think I would just repeat, I would like people to remember that their carefully accumulated clinical experience and doing what I call practical clinical science, taking careful note of the sorts of things that we were talking about before, is more valuable than an RCT and that they mustn't be intimidated by guidelines. I'll try and remember to find guidelines to

[00:23:57] guidelines paper. The other paper I'd recommend to people is John Ioannidis. Anybody listening who doesn't know any of John Ioannidis' work, he's written a huge number of papers. He's got an H-index of 10,000 or something. It's mind-hoggling. Wow. I'm joking. It's 500 or 700. It's absolutely crazy. But a lot of it, most of it, I'm sure, is very good stuff. And he wrote a wonderful paper, Why Most Medical Research

[00:24:27] is Wrong. I will definitely look that up. Look, the question I've often said to people is to focus on the reality that so much research is useless. Go back to what you thought 10 years ago. You thought you were keeping up to date and you were taking notice of all this stuff and you were trying this, that, and the other. How many of those things are you still doing and how many of those things do you still think are worth doing? Well, apply the same thing

[00:24:57] to what you're doing at the moment because you're doing what you said, Ethan, you're probably latching on to all the new stuff, falling for all the bullshit, and engaging in activities which, in fact, don't actually have substantive validity behind them. And working with low probabilities of getting your patients better. Yep. All right. Well, hey, it's been a lot of fun. I'm exhausted. I'm ready for a glass of champagne. I'm ready to go to bed about, but one of these days, you know, my wife and I really want to come.

[00:25:27] She's been to Australia before. We want to come back. We might just have to come and have a glass of champagne with you. That sounds a great idea. We'll find you a crocodile. Perfect, perfect. All right, Ken, thank you so much. You have a good rest of your day, all right? Bye, folks. Somebody get this guy some help. Disclaimer, this podcast is for

[00:25:57] informational purposes only. The information provided in this podcast and related materials are meant only to educate. This information is not intended as a substitute for professional medical advice. While I am a medical doctor and many of my guests have extensive medical training and experience, nothing stated in this podcast nor materials related to this podcast, including recommended websites, texts, graphics, images, or any other materials should be treated as a substitute for professional medical or psychological advice, diagnosis, or treatment. All listeners should consult with a medical professional, licensed mental health provider, or other healthcare provider if seeking medical advice, diagnosis, or treatment. Or, put more simply, if you need help like this guy, call your

[00:26:27] own doctor. Thanks again for watching and or listening. If you're passionate about the subjects that I discuss on the channel, do me a favor and like, comment, subscribe, do whatever you can to make your voice heard, that these are problems that must be addressed in our society. If you have any questions, comments, or concerns, I want to hear them. Feel free to reach out on

[00:26:56] social media or email us at renegadesych at gmail dot com. And if you'd like to be a guest of the show or you have a connection to somebody that you think would be a good guest, let us know. Thanks again for listening.