Hello MikeB:
Very interesting input from all. A quick question for you when you get a minute if you don't mind answering. I was not aware that there is a major issue with research not being published. I know that I have a read many abstracts, but I was not aware that a researcher could potentially selectively hold back information. Generally, sometimes we know there is something out there that we can try to research, but there are other "unknowns" that we are ignorant about.
I know that trials get cut off for a number of reasons. Assuming one of the reasons that I have come across, a trial gets terminated because data from in the trial or even from outside of the trial kills the drug or the question of the trial; then I would understand that the project has been killed. It would take a considerable effort, potentially, to generate the worthless final report, so they just terminate the effort.
If you generate a report for a scientific learned publication or an established meeting, the report has to be significant, requires a lot of analysis, and needs to stand up to peer review. For such a reason, and similar reasons, I would understand results being held back, so that resources can be expended on more worthwhile efforts.
On the other hand, if a large company or hospital has performed a trial, and they are holding back some information for "competitive" reasons, to me that would be a bad thing that, and as you say, should be reformed.
Can you briefly explain your understanding of the process, or is there an article or source that explains that?
Thank you very much and best regards, JPC
Forums
Re: Medical research not always what it should be
Hi Everyone!
This is an interesting thread, albeit gloomy, since survival stats for myeloma are still not over 60% for a 5 year survival, on average, in any country. (Correct me if I am wrong on that).
There are many factors that can go into survival, including the availability of the most recent treatments, and the availability of supportive health care. It does no good to have new treatments, if they cannot be accessed by patients. Some countries lag behind getting the latest treatments, but it is my hope that eventually there will be a more level playing field for everyone who has myeloma.
The chromosomal risk factors also should be taken into account, for actually there are many types of myeloma, according to genetic risk. Many times, the only access a patient may have to a new drug not yet approved in their country would be through clinical trial. We know that not everyone can even qualify to be on a clinical trial. Myeloma treatments continue to evolve and hopefully we will see the survival rates improve everywhere.
This is an interesting thread, albeit gloomy, since survival stats for myeloma are still not over 60% for a 5 year survival, on average, in any country. (Correct me if I am wrong on that).
There are many factors that can go into survival, including the availability of the most recent treatments, and the availability of supportive health care. It does no good to have new treatments, if they cannot be accessed by patients. Some countries lag behind getting the latest treatments, but it is my hope that eventually there will be a more level playing field for everyone who has myeloma.
The chromosomal risk factors also should be taken into account, for actually there are many types of myeloma, according to genetic risk. Many times, the only access a patient may have to a new drug not yet approved in their country would be through clinical trial. We know that not everyone can even qualify to be on a clinical trial. Myeloma treatments continue to evolve and hopefully we will see the survival rates improve everywhere.
-
Nancy Shamanna - Name: Nancy Shamanna
- Who do you know with myeloma?: Self and others too
- When were you/they diagnosed?: July 2009
Re: Medical research not always what it should be
Hello again Ian.
My aim is not to start WW3 with this post, but to raise awareness. I am interested to know if you have been a medical / scientific researcher with inside knowledge of how things actually can happen in a very competitive scientific arena?
As to survival statistics in the UK, these are worse for most common cancers than in other wealthier countries. But it seems that this is not necessarily so now with myeloma. You are correct how statistics is presented can be misleading and often comparisons are not like for like or cover the same time periods.
But I recall the outburst from an ex-oncology researcher who I worked with- when I told her
I had myeloma she stated she had known two people with it in the late 1990'a who died within 3 months. It is obvious that is how she saw my survival possibility, two years ago. It did not lead to ongoing contact.
But as I state regarding the reported improved myeloma survival figures, in the UK things will go backwards with politicians / government calling for faster diagnosis of cancers, yet denying treatments based on cost under a cloak of not being 'cost effective'.
The cancer charities have their own agendas too as institutions. We as patients do not feel as powerful as US ones and are aware that for those well insured in the US options are greater as 'consumers'. But in the UK those with medical insurance or private means too will have access to treatments others in the national system will be denied.
Nancy- your determination and optimism about treatments on an equal level for all myeloma patients is heartwarming. But I take things further than my myeloma. I would wish to see medical and health care readily accessible to anyone who develops any illness through bad luck. I would not stop at myeloma as human illness does not come with a single label.
Best wishes to all
My aim is not to start WW3 with this post, but to raise awareness. I am interested to know if you have been a medical / scientific researcher with inside knowledge of how things actually can happen in a very competitive scientific arena?
As to survival statistics in the UK, these are worse for most common cancers than in other wealthier countries. But it seems that this is not necessarily so now with myeloma. You are correct how statistics is presented can be misleading and often comparisons are not like for like or cover the same time periods.
But I recall the outburst from an ex-oncology researcher who I worked with- when I told her
I had myeloma she stated she had known two people with it in the late 1990'a who died within 3 months. It is obvious that is how she saw my survival possibility, two years ago. It did not lead to ongoing contact.
But as I state regarding the reported improved myeloma survival figures, in the UK things will go backwards with politicians / government calling for faster diagnosis of cancers, yet denying treatments based on cost under a cloak of not being 'cost effective'.
The cancer charities have their own agendas too as institutions. We as patients do not feel as powerful as US ones and are aware that for those well insured in the US options are greater as 'consumers'. But in the UK those with medical insurance or private means too will have access to treatments others in the national system will be denied.
Nancy- your determination and optimism about treatments on an equal level for all myeloma patients is heartwarming. But I take things further than my myeloma. I would wish to see medical and health care readily accessible to anyone who develops any illness through bad luck. I would not stop at myeloma as human illness does not come with a single label.
Best wishes to all
Re: Medical research not always what it should be
Hello again,
This thread is generating some good discussion and debate. Hopefully we will avoid WWIII, as Edna says!
It's clear that participants in this thread hold some strong opinions, which is good. Even if, or maybe I should say especially if, those opinions are not in agreement with one another. I learn the most when people question a position I've taken or statement I've made, and I need to back up what I've said or reconsider my position.
As a little aside, the R&D institution (in the telecommunications industry, BTW, not medicine) where I spent most of my career was famous internally for an odd cultural thing. If you were giving a presentation or trying to push a position during a meeting and someone said to you, "I have a *dumb* question ..." you knew you were in trouble because the question was almost always anything but dumb. The questioner had likely spotted a key flaw in your experiment or your logic. Over the years I learned a lot from those "dumb" questions. So I love these kinds of discussions in The Beacon where we question one another like this - respectfully, of course.
That said, one of my strong opinions related to this thread is that results from all clinical trials should be made public, as I stated in an earlier post. That opinion has led to some good comments and questions.
Rneb, you are right about HIPAA. For those outside the US, HIPAA stands for the Health Insurance Portability and Accountability Act, enacted in the US in 1996. It protects the privacy of a patient's health information. I'm not an expert on HIPAA, despite needing to read and sign many forms about HIPAA over the years.
But in my call for clinical trial data to be made available at the individual participant level (as well as summary data level, of course), I'm assuming that anything that would allow for the identification of the individual patient would be masked. And I know we could get into a big discussion about what exactly that means when you get down to individual datum, but I think that's beside the point of this thread. BTW, I hate butter brickle ice cream, Rneb, so that's one data point you can use to help identify me.
JPC, you asked a good question about the scope of the problem of unpublished clinical trial research. And you made some good points about how researchers would typically want to move on to something more promising as quickly as possible, if they run into negative results and a trial is stopped early (without taking the time to "write up" the negative results).
But it's estimated that as many as 50% of completed trials do not get get published. Ross, et al (2010) looked at a random sample of 635 clinical trials completed by Dec 31, 2008, and found that only 46% were published in a peer-reviewed journal indexed by Medline within 30 months of the completion date of the trial.
Bourgeois, et al (2010) found that overall 66% their random sample of trials registered in ClinicalTrials.gov had published results within 24 months of the completion of the trial. But only 32% of industry-funded trials were published in that time period. And of the industry-funded trials that were published, 85% had positive results, while only 50% of government-funded trials that were published had positive results.
It is important that negative results, as well as positive results are published. At a macro level we can learn just as much from negative results as from positive results because we learn what paths not to pursue. Remember Thomas Edison's quote about the many failures he endured on the way to the incandescent light bulb: "I have not failed. I've just found 10000 ways that won't work."
A close-to-home case of experimental "failure" is the finding by Golden et al (2009) that green tea does not enhance the effectiveness of bortezomib (Velcade), but rather interferes with it. Fortunately for us multiple myeloma patients who have used bortezomib, these results that were completely counter to the researchers' hypotheses were published.
At a micro level, there may be even more to learn from many clinical trials that are deemed to be failures when you look in detail at a few "exceptional responders" who may have done well with the trial drug, rather than washing out their results by averaging across all trial participants. That's why publishing results down to the individual participant level is important.
I hope my comments here help clarify why I think there is a problem with many clinical trial results not being made public, as I think they should be. I should emphasize, though, that I'm just stating my opinion and trying to defend it. I'm open to hearing other positions on this issue, too.
Mike
Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010 Aug 3;153(3):158-‐166. doi:10.1059/0003-‐4819-‐153-‐3-‐201008030-‐00006.
Golden EB, Lam PY, Kardosh A, Gaffney KJ, Cadenas E, Louie SG, Petasis NA, Chen TC, Schonthal AH. Green tea polyphenols block the anticancer effects of bortezomib and other boronic acid-based proteasome inhibitors. Blood. 2009;113:5927–5937.
Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012 Jan 3;344:d7292. doi: 10.1136/bmj.d7292.
This thread is generating some good discussion and debate. Hopefully we will avoid WWIII, as Edna says!
It's clear that participants in this thread hold some strong opinions, which is good. Even if, or maybe I should say especially if, those opinions are not in agreement with one another. I learn the most when people question a position I've taken or statement I've made, and I need to back up what I've said or reconsider my position.
As a little aside, the R&D institution (in the telecommunications industry, BTW, not medicine) where I spent most of my career was famous internally for an odd cultural thing. If you were giving a presentation or trying to push a position during a meeting and someone said to you, "I have a *dumb* question ..." you knew you were in trouble because the question was almost always anything but dumb. The questioner had likely spotted a key flaw in your experiment or your logic. Over the years I learned a lot from those "dumb" questions. So I love these kinds of discussions in The Beacon where we question one another like this - respectfully, of course.
That said, one of my strong opinions related to this thread is that results from all clinical trials should be made public, as I stated in an earlier post. That opinion has led to some good comments and questions.
Rneb, you are right about HIPAA. For those outside the US, HIPAA stands for the Health Insurance Portability and Accountability Act, enacted in the US in 1996. It protects the privacy of a patient's health information. I'm not an expert on HIPAA, despite needing to read and sign many forms about HIPAA over the years.
But in my call for clinical trial data to be made available at the individual participant level (as well as summary data level, of course), I'm assuming that anything that would allow for the identification of the individual patient would be masked. And I know we could get into a big discussion about what exactly that means when you get down to individual datum, but I think that's beside the point of this thread. BTW, I hate butter brickle ice cream, Rneb, so that's one data point you can use to help identify me. JPC, you asked a good question about the scope of the problem of unpublished clinical trial research. And you made some good points about how researchers would typically want to move on to something more promising as quickly as possible, if they run into negative results and a trial is stopped early (without taking the time to "write up" the negative results).
But it's estimated that as many as 50% of completed trials do not get get published. Ross, et al (2010) looked at a random sample of 635 clinical trials completed by Dec 31, 2008, and found that only 46% were published in a peer-reviewed journal indexed by Medline within 30 months of the completion date of the trial.
Bourgeois, et al (2010) found that overall 66% their random sample of trials registered in ClinicalTrials.gov had published results within 24 months of the completion of the trial. But only 32% of industry-funded trials were published in that time period. And of the industry-funded trials that were published, 85% had positive results, while only 50% of government-funded trials that were published had positive results.
It is important that negative results, as well as positive results are published. At a macro level we can learn just as much from negative results as from positive results because we learn what paths not to pursue. Remember Thomas Edison's quote about the many failures he endured on the way to the incandescent light bulb: "I have not failed. I've just found 10000 ways that won't work."
A close-to-home case of experimental "failure" is the finding by Golden et al (2009) that green tea does not enhance the effectiveness of bortezomib (Velcade), but rather interferes with it. Fortunately for us multiple myeloma patients who have used bortezomib, these results that were completely counter to the researchers' hypotheses were published.
At a micro level, there may be even more to learn from many clinical trials that are deemed to be failures when you look in detail at a few "exceptional responders" who may have done well with the trial drug, rather than washing out their results by averaging across all trial participants. That's why publishing results down to the individual participant level is important.
I hope my comments here help clarify why I think there is a problem with many clinical trial results not being made public, as I think they should be. I should emphasize, though, that I'm just stating my opinion and trying to defend it. I'm open to hearing other positions on this issue, too.
Mike
Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010 Aug 3;153(3):158-‐166. doi:10.1059/0003-‐4819-‐153-‐3-‐201008030-‐00006.
Golden EB, Lam PY, Kardosh A, Gaffney KJ, Cadenas E, Louie SG, Petasis NA, Chen TC, Schonthal AH. Green tea polyphenols block the anticancer effects of bortezomib and other boronic acid-based proteasome inhibitors. Blood. 2009;113:5927–5937.
Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012 Jan 3;344:d7292. doi: 10.1136/bmj.d7292.
-
mikeb - Name: mikeb
- Who do you know with myeloma?: self
- When were you/they diagnosed?: 2009 (MGUS at that time)
- Age at diagnosis: 55
Re: Medical research not always what it should be
Hi Mikeb
You make a very cogent case for your position, one that I would support.
Peer reviewed research is not always quite as 'clean' as some think. It is people in the same field, research competitors, who are usually asked to review a paper for publication. that can bring conflict of interest into play.
I think I would like to see raw data from research undertaken at one institution, analysed in depth by another of the same standing or higher with publications coming out as a joint effort from independently analysed information. This might avoid the bias in reporting that can arise when one institution analyses its own data.
Although unrealistic, I would like to see research undertaken less in silos competing with each other but to corroborate / consolidate findings, or otherwise, and to answer questions that arise from published research.
There just is not enough money for everyone around the world to be doing much the same without working together to raise the quality and to enable money to be invested in actually treating patients using the most up to date information from high quality research data. In any field there will be 'blind alleys' taken in the process, but it is the questions these raise about the 'next avenue to explore' that are important. So negative findings can help other researchers take a different approach.
Edna
You make a very cogent case for your position, one that I would support.
Peer reviewed research is not always quite as 'clean' as some think. It is people in the same field, research competitors, who are usually asked to review a paper for publication. that can bring conflict of interest into play.
I think I would like to see raw data from research undertaken at one institution, analysed in depth by another of the same standing or higher with publications coming out as a joint effort from independently analysed information. This might avoid the bias in reporting that can arise when one institution analyses its own data.
Although unrealistic, I would like to see research undertaken less in silos competing with each other but to corroborate / consolidate findings, or otherwise, and to answer questions that arise from published research.
There just is not enough money for everyone around the world to be doing much the same without working together to raise the quality and to enable money to be invested in actually treating patients using the most up to date information from high quality research data. In any field there will be 'blind alleys' taken in the process, but it is the questions these raise about the 'next avenue to explore' that are important. So negative findings can help other researchers take a different approach.
Edna
Re: Medical research not always what it should be
Hello Mike:
Thank you very much for your very well written post. If the level of studies where they hold back data is in the range of 50%, then it very well may be that something is rotten in the state of Denmark. Considering your information and my previous position, if there were minimum reporting requirements where study authors or sponsors had to provide a minimal amount of data in a closeout report, I certainly agree that would make sense, or the entity should not be approved for the next study. That minimum level of data provided could start at a level of effort as your post, which actually read as a short published article.
On the other side of the ledger, I would be concerned with new requirements to add major costs to the system, which is already very costly. There are very good reasons for many of the existing requirements, however, I have come across leading researchers (who actually publish results) that have suggested there is opportunity to streamline the process for promising new drugs.
Again, thank you for replying to my question with your excellent post. Best Regards, JPC
Thank you very much for your very well written post. If the level of studies where they hold back data is in the range of 50%, then it very well may be that something is rotten in the state of Denmark. Considering your information and my previous position, if there were minimum reporting requirements where study authors or sponsors had to provide a minimal amount of data in a closeout report, I certainly agree that would make sense, or the entity should not be approved for the next study. That minimum level of data provided could start at a level of effort as your post, which actually read as a short published article.
On the other side of the ledger, I would be concerned with new requirements to add major costs to the system, which is already very costly. There are very good reasons for many of the existing requirements, however, I have come across leading researchers (who actually publish results) that have suggested there is opportunity to streamline the process for promising new drugs.
Again, thank you for replying to my question with your excellent post. Best Regards, JPC
-
JPC - Name: JPC
Re: Medical research not always what it should be
Hi Edna and JPC,
Just a quick note to say I strongly agree with what both of you said in your latest posts in this thread.
To your point, Edna - While some might think of it as a "race" to find a cure for multiple myeloma, I think think that's not such a great analogy because of the competitive nature of a race. You're entirely right that limited resources make duplication of effort especially wasteful and problematic. Better sharing of information and better cooperation go hand in hand.
And, JPC, you're right that it would be good to have some low-effort way to share negative results. It does take a lot of work to right up a paper for publication. I'm imagining some on-line repository for short summary reports for those cases, along with a public database to store the data that were generated.
Maybe something like this concept for sharing clinical trial results and data will emerge that's in the same spirit as open source software, Wikipedia, etc. But I'm not holding my breath, mainly because of the big bucks involved in medical research.
Mike
Just a quick note to say I strongly agree with what both of you said in your latest posts in this thread.
To your point, Edna - While some might think of it as a "race" to find a cure for multiple myeloma, I think think that's not such a great analogy because of the competitive nature of a race. You're entirely right that limited resources make duplication of effort especially wasteful and problematic. Better sharing of information and better cooperation go hand in hand.
And, JPC, you're right that it would be good to have some low-effort way to share negative results. It does take a lot of work to right up a paper for publication. I'm imagining some on-line repository for short summary reports for those cases, along with a public database to store the data that were generated.
Maybe something like this concept for sharing clinical trial results and data will emerge that's in the same spirit as open source software, Wikipedia, etc. But I'm not holding my breath, mainly because of the big bucks involved in medical research.
Mike
-
mikeb - Name: mikeb
- Who do you know with myeloma?: self
- When were you/they diagnosed?: 2009 (MGUS at that time)
- Age at diagnosis: 55
Re: Medical research not always what it should be
A few years ago when I been initially diagnosed I had a friend who was close friends with a PhD myeloma specialist actively doing research into a treatment – and hopefully a cure – for multiple myeloma at a major university. My friend said their researcher friend was very concerned about their research leaking out before it was published because somebody else could appropriate it as their own and publish it before they did. It is sad to think that this kind of thing has to be a concern and what it does to inhibit progress in curing diseases like cancer.
Edna and JPC- Sharing of scientific data, peer review of that data, and independent duplication of results are key attributes of the scientific method. But as you have observed, this is not always the case in medical research.
Mike and JPC – I second the sharing of negative results. It would be helpful if negative results were openly shared. I feel there often is potential to learn a lot from why real world experimentation or a trial did not match theory and key insights into the nature of, or mechanism of, how a system works can often be made. Many examples of this in scientific research can be found. Has there been a problem with negative data or failed trial results being withheld or “buried” and not made public?
Edna and JPC- Sharing of scientific data, peer review of that data, and independent duplication of results are key attributes of the scientific method. But as you have observed, this is not always the case in medical research.
Mike and JPC – I second the sharing of negative results. It would be helpful if negative results were openly shared. I feel there often is potential to learn a lot from why real world experimentation or a trial did not match theory and key insights into the nature of, or mechanism of, how a system works can often be made. Many examples of this in scientific research can be found. Has there been a problem with negative data or failed trial results being withheld or “buried” and not made public?
-
Eric Hofacket - Name: Eric H
- When were you/they diagnosed?: 01 April 2011
- Age at diagnosis: 44
Re: Medical research not always what it should be
Here's some news from today about an especially egregious example of falsified data in a cancer clinical trial.
"Scientist falsified data for cancer research once described as ‘holy grail,’ feds say", Washington Post, Nov 9, 2015 (full text of article)
Excerpt:
I'd heard of this case awhile ago; the federal report investigating what happened was just released.
This episode supports Edna's original concerns at the top of this thread.
Mike
"Scientist falsified data for cancer research once described as ‘holy grail,’ feds say", Washington Post, Nov 9, 2015 (full text of article)
Excerpt:
One Duke University surgeon called it a “new frontier” in cancer treatment. Another said it could save “10,000 lives a year” or more. A researcher at Mass General Hospital called it “a very, very exciting tool” in the fight against lung cancer. As news spread in 2006 and 2007 of the work of Anil Potti, a star cancer researcher at Duke, the excitement grew.
What he had claimed to achieve, in leading medical journals, was a genomic technology that could predict with up to 90 percent accuracy which early stage lung cancer patients were likely to have a recurrence and therefore benefit from chemotherapy ...
Now, the Office of Research Integrity (ORI), the agency that investigates fraud in federally-funded medical research, has officially declared that the data generated by Potti was not only flawed, but “false.”
I'd heard of this case awhile ago; the federal report investigating what happened was just released.
This episode supports Edna's original concerns at the top of this thread.
Mike
-
mikeb - Name: mikeb
- Who do you know with myeloma?: self
- When were you/they diagnosed?: 2009 (MGUS at that time)
- Age at diagnosis: 55
Re: Medical research not always what it should be
A comment on MikeB's post.
The Washington Post is still a respected, major news outlet. The press is not perfect, and does frequently make mistakes. So generally we should not overreact to a single news article.
However, ...., FALSIFYING an approved clinical study, intentionally (as opposed say to just making a wrong conclusion) is something in my view that someone should GO TO JAIL.
I would be interested as to what happened to this doctor after the "error" had been established.
Thanks for the very interesting information. Best Regards,
The Washington Post is still a respected, major news outlet. The press is not perfect, and does frequently make mistakes. So generally we should not overreact to a single news article.
However, ...., FALSIFYING an approved clinical study, intentionally (as opposed say to just making a wrong conclusion) is something in my view that someone should GO TO JAIL.
I would be interested as to what happened to this doctor after the "error" had been established.
Thanks for the very interesting information. Best Regards,
-
JPC - Name: JPC
21 posts
• Page 2 of 3 • 1, 2, 3