Blog Archives

7 Site Selection Strategies for Your Next Clinical Trial

Selecting the right sites can be the difference in having a study that runs with a smooth operational flow or a study filled with rescue sites and their associated delays. We have come up with a list of seven strategies you can employ the next time you are identifying the best sites for your trial.

Take inventory

Having a clear sense of what you are seeking can greatly guide your site selection.  Do you know how many sites you need overall, by country, and how much room you have for adjustment?  For example if you realize you can recruit more subjects in a given country, can you allocate more resources away from one country to the promising country?

Know the metrics

Selecting sites without understanding the site metrics for your trial is not an advisable strategy.  Maybe you know you need to find sites in France for your upcoming Phase II Lung Cancer study.  If you knew the site metrics then you would know that at any given site in France you could expect to randomize 3 subjects.  Later if a site in France was saying they could provide 10 subjects, you would be able to question their assumptions.

Internalize your internal priorities

Is it important for your company (or this trial) to use sites that you have worked with in the past? Are relationships more important that the performance of the site?  While it may be common to work with several sites who are average performers, is your organization open to working with new sites instead? There is always a balance of trying to use the sites with the most promising performance, with those sites that are reliable for being average – this is where knowing the priorities of your company come in to play. What policies have informed past decisions and is there room to update these as you progress your site selection strategy?

New site thinking

Using a site that is new may involve more work at the outset, but will be well worth the effort for this study and any possible future collaborations.  With new cross-industry developments aiming to minimize the administrative burden for both sites and sponsors and CROs, choosing to work with a new site can be an excellent decision.  Use individual site scorecards to inform your site selection by assessing the site’s performance in past trials for your same disease.

Experience or performance or both

Some sites have literally dozens of trials in the area of your study.  Other sites may have an n of 1.  Or perhaps no experience in your indication but experience in the same therapy area (TA) or disease area (DA).  When choosing a site, do you want to choose sites with a proven track record in your indication?  We will call this “experience” – an experienced site for your indication.  However this does not account to how well the site performed.  Some sites worth considering may be excellent performers at the TA or DA level, despite the fact they do not have experience (yet) in your indication.  With the right information you can aim to find the sites that have both an ideal level of experience and performance.

Manage risk

Going back to our example of our site in France saying they could provide 10 subjects when you know the median performance at any site in France is 3.  When in discussions with that site you could learn why they think they could provide you that many subjects.  You could mitigate risk by planning for fewer subjects. View that site’s particular performance for past lung cancer studies to know their median performance for the Industry to further guide your planning.

Be protocol specific   

Trials have different resources meaning the basis you use for assessing site performance, and thus your site selection, can be quite different for each trial.  If recruiting a large number of subjects is the highest criteria, keep this in mind as you evaluate historical site performance.  Or maybe your priorities include a balance of the time it takes for patient randomization and the number of subjects the site it can recruit.  You want (need) sites that can randomize in a respectable time and with a reliable number of subjects.  Knowing the unique priorities of the trial can help you to select sites that are best suited to deliver for your trial.

The next time you are identifying the sites for your upcoming trials, use one or all of these seven strategies.  The more informed the better.  Wishing you smooth operational flow in your site selection.


R&D Spending & Revenues For Cancer Drugs

We read with interest a recent article (“Research and Development Spending to Bring a Single Cancer Drug to Market and Revenues After Approval”) published in JAMA Internal Medicine by Prasad and Mailankody and wanted to provide our perspective on this important topic.

We at KMR Group have been working with the R&D pharmaceutical industry for over 25 years and have invested significant time helping companies understand how much they are spending to develop products, how they compare to the broader industry, and identifying opportunities for improvement in their organizations.

While the article was helpful for raising crucial questions about pharma R&D, there were serious problems with the methodology, leading the authors to understate the true cost of drug development and exaggerate the return on investment.

One of the most egregious flaws with the method was the sampling of companies and products. The study picked small companies with no prior approvals, that had an extremely novel NME approved, and then extrapolated this data to all of pharma R&D.

Why is this sampling a problem?

The authors claim that because these companies were simultaneously developing several compounds the cost of failure was included. This is misleading. Using their data, 10 companies had 43 drugs in development, of which 12 were approved, a 28% success rate. By setting the criteria to include only companies that were successful, the success rates are skewed.

In terms of success rates, a critical metric for pharma, we have proprietary data that is carefully collected directly from pharma companies. This data is treated consistently across the companies and carefully reviewed to ensure accuracy. Indeed, we recently reported1 that in 2012-16 the overall NME success rate was 6% (i.e., 6% of new drugs entering Preclinical reach approval). This is for all diseases, not just Oncology; Oncology is close to that figure. This is far from the 28% rate in the article. A reliable spending figure must properly account for the cost of failure; the JAMA article fails to adequately account for this.

Second, the focus is on Oncology, where there has been much change over the last decade. The fruits of labor have finally begun to benefit the industry after long frustration, but all the prior work and cost is not accounted for here. We liken this to a present day search for Alzheimer’s drugs. Companies have been investing billions in search for a breakthrough Alzheimer’s treatment, but as of yet have seen minimal return. If a scientific breakthrough happens in the next 5 years and the market expands with novel medicines, it would be misleading to only take the last 5 years of effort when assessing the R&D return.

There are other major issues with the methodology. Despite discounting the cost of failure inappropriately, even for the companies tracked there were significant drug costs missing:

Licensing/Acquisition costs: Half of the drugs (5 of 10) originated outside the companies. For these drugs, costs prior to this point were excluded.

Post-approval commitments: At an increasing rate, regulatory agencies are working with pharmaceutical companies to approve drugs but mandate that follow-up trials and programs are run even after the drug is marketed. None of these post-approval commitments are included in the expenses.

Discovery work: The costs required to bring a drug from the lab to the clinic are not included in the analysis. The authors do try and capture preclinical costs, but designate start of R&D to be 2 years before initial mention in literature. This is not sufficient time to go from target identification to first dose in humans.

Co-Development: 60% of the drugs had a collaborative agreement with another pharmaceutical company. The authors state that for these drugs, R&D expenditures from both companies were included, but the method is unclear. For example, Iburtinib was co-developed between Pharmacyclics and Johnson & Johnson. It is not clear what figures were added to the Pharmacyclics R&D spending to account for J&J’s portion.

The other main element the article focused on was the revenues after the drugs were approved.

The selection bias of these companies also had a significant effect when looking at sales. Half of the products were acting on novel targets and had expedited development timelines, which not only reduce cycle times and overall spending to bring the drug to market, but inflate the revenues as well. These were novel medicines and so there is no surprise the revenues would be higher when compared to more standard developed drugs.

In addition, to bolster sales companies expend significant sums on marketing, education, and awareness of their products. Without these expenses (which are not captured in the cost calculations) the drugs would not have such high returns.

Finally, while some drugs do generate significant returns, there is no comparison to the alternative therapies or treatments. Novel drugs can be expensive but how do they compare to the other options? It is possible that even expensive drugs are reducing overall healthcare costs.

We are not the only firm to find criticism with their findings. The New York Times shared views of other Industry leaders in response to the article.

We want to end on a question that characterizes the conclusion of the article. Could you develop a cancer drug with $648 million? If so, would it return 10x the investment? It’s possible but we wouldn’t bet on it. To have an accurate sense of the cost of drug development requires access to the most reliable data, careful analysis, and proper method.

1Pharmaceutical Benchmarking Forum 2017 R&D Performance: Success Rates & Cycle Time, KMR Group, June 2017


How Long to Get 50%, 75%, 90% of Sites Up and Running…It Depends

Yes, it depends. We have been getting the question of whether we benchmark just how long it takes to get 50, 75 and 90 % of sites up and running. KMR first performed this analysis back in 2011 for our inaugural Enrollment Productivity Study and then in 2014 as part of our Enrollment Insights program.

Given this analysis is now going mainstream, we are keen on understanding how companies are planning to use this data. We have also revisited this analysis and have a few thoughts that might help the discussion.

From a technical aspect, the analysis is fairly straightforward. However, the interpretation can be difficult because this type of analysis uses percentiles which can mask issues related to study size. For the same reason it can be tricky to assess performance or set benchmarks. Even within a specific indication the results may be skewed because of the varying sizes of studies. For example, if your company runs larger studies on average, your time will likely be longer to get to a percentile of sites initiated, but is that fair? For example, a study with 10 sites vs a study with 100 sites will have very different cycle times to get to the 50th percentile of sites initiated.

We tend to think of this metric as an “on-study” metric, one that is helpful to assess during the course of a study to help those during study conduct determine whether a given study is “on target”. For these types of analyses the percentile and the associated months are generally tied to a plan and these plan targets are determined based to some extent on the disease and size of study. But the question is “what is the best metric to set the plan?”. Instead of the “time to achieve the 50th percentile of sites initiated” we would advocate using a site initiation rate benchmark. To get more nuanced you might set this rate based on the percentile intervals, e.g., the site initiation rate benchmark from the 0-25th percentile. Indeed, we have found that the rate of site initiation is considerably lower in the last quartile compared to the first quartile of the site initiation period. Also, using a rate rather than the actual time to the specific percentile helps to eliminate the bias of study size either positive or negative.

Now that we have shared some of our thoughts around this type of analysis, we would be interested in hearing yours. Can you tell us how you plan to use this type of performance measure? Is your interest geared more towards planning or assessing performance, e.g., how long it takes to get x% of sites initiated for purposes of planning a new study? Or more along the lines of whether your company is getting its sites initiated efficiently? Or something completely different! Let us hear from you and we may print your response.

Now that we have presented this information, we are excited to let you know that our Enrollment Productivity Study is on the schedule for 2018 so you can expect to hear more about this analysis soon.

Linda Martin
President and Founder
KMR Group


Congrats to Novartis Leadership

KMR Group wants to lend a shout out on the appointment of Vasant (Vas) Narasimhan being named as CEO of Novartis.  We congratulate him on achieving such an honor and also for retiring CEO Joseph Jimenez’ years of service.  It is clear from the tone of the press release and words from the chairman of the board Joerg Reinhardt, retiring CEO Joseph Jimenez, and Vasant Narasimhan that the transition is one of mutual respect, unusual for this day and age and so befitting what we would expect from such high ranking executives.  Bravo Novartis for not only developing world class medicines but for also setting an example on how to tastefully pass the torch.


Trial Cost Gets Hot

I could not resist the temptation to title this blog post “Trial Cost Gets Hot” given it’s August and temperatures are soaring in places like San Antonio Texas and Death Valley (where by the way you can jump into an all-natural “hot springs” pool to “cool” off. But to jump right in to this topic, I want to discuss how pleased we were to publish some top line results on the cost of clinical trials recently in Nature Review Drug Discovery. At the time of authoring the article, I didn’t realize it would stir such strong opinions and reactions about how much trials cost, or at least how much we think they cost


But whatever your current understanding of the cost of clinical trials, we believe the time is right for companies to start measuring and analyzing the cost of clinical trials and to begin more thoughtful dialogue on what it means….what it means to be a high cost company; or what it means to be an efficient company.

The bottom line is that wherever your company sits on this pendulum is only the start of the discussion and the interpretation depends on what question you are trying to answer or what your company is trying to achieve.

Benchmarking cost at an individual trial level takes a bit of effort and maybe even a little trial and error, but when done right, it can be an extremely powerful tool in shaping and driving performance. Indeed, cost is a lever in not only efficiency, but also in getting drugs to market faster.

Only by having access to this type of reliable cost data using a framework that can help the discussion can you start to gain insight into differing strategies and their results.

We start our next round of Trial Cost this month. Hope you can join us.

Carpe diem!

Linda Martin
President and Founder
KMR Group


We Started A Blog

I am pleased to start this blog for KMR Group. It’s an opportunity for me and others at KMR Group to share some inside thoughts on our work….thoughts that may never make it into a publication or a report. And to address questions or add to the dialogue on a variety of performance or analytics based topics from R&D productivity to site performance possibly in some far flung corner of the world.

We hope these short, concise ideas add to your understanding of R&D performance.

Carpe diem!

Linda Martin
President and Founder
KMR Group