Acting Director Bruce Reed posted a blog post on the consolidation of all NIH grant review within the Center for Scientific Review (CSR). There are some tidbits of interest including:
In 2024, CSR reviewed 66,697 applications….With consolidation, CSR became responsible for managing the review of about 30,000 additional applications per year.
How did they accomplish this? Well, by leaning on the people previously doing the job within the IC study sections.
CSR expects to create 52 new chartered study sections to manage the review of R01 applications previously handled by funding ICs. These study sections are essentially being created out of existing study sections, which are now oversubscribed. Overflowing study sections will be divided to cover different, scientifically coherent sets of topics.
CSR is recruiting reviewers who had previously served on panels convened by funding ICs to retain valuable expertise…Most of the review staff who were at IC branches are now staffing CSR meetings.
Pretty reasonable stuff. This apparently says they are drawing in experienced SROs and the same super set of potential grant reviewers to cover the same old topics.
So far, so good.
There’s some blah-de-blah about making review more consistent via minimizing “significant local variation in review practices” without specifying what that may mean. Standard guidelines….training…yeah yeah, sure.
Here’s the howler slash head scratcher. One of those things that makes you wonder if possibly the longest running, most experienced and most senior person left at the CSR has any idea how grant review and award actually works.
Having review meetings with applications grouped according to science, rather than the IC that administers them, promotes appropriate competition. Better competition should ultimately result in better review outcomes.
Huh?
Let’s see how this works with some closely-related ICs and study sections pulled totally at random.
We already know that the supposedly already-in-progress ENQUIRE process reviewed NMB, NAL and BRLE this past cycle, and decided to keep BRLE as-is, but to revamp NMB and NAL. From my point of view of having a lot of proposals reviewed in BRLE, some in NMB and a handful in NAL, as well as being empaneled for a term of service on BRLE way back when, I cannot help drawing the direct connection.
NAL stands for Neurotoxicology and Alcohol but it has always, to my limited view, had a big domination of alcohol-related proposals that would be funded by NIAAA. In fact I once learned the hard way that this panel did not in fact actually have interest in non-alcohol neurotoxicology. NMB (Neurobiology of Motivated Behavior) has typically been dominated by proposals that would be funded by NIDA. BRLE, at least in times past, has a bit more of a diverse mission. When I was on it we seemingly got NIDA and NIAAA directed proposals, as well as a fair number from other ICs such as NIMH, NICHD, NINDS, what have you.
NIAAA also has maintained AA4 (NEUROSCIENCE AND BEHAVIOR) as an in-house study section. I never was very clear on why they did so and why some proposals would be reviewed there instead of, say, BRLE, NMB or NAL.
I don’t think NIDA had any lasting generalist panels like AA4, but they sure did have SEPs convened for targeted FOA/NOFO. It probably worked out about the same.
There is a certain species of my colleagues, and I love and sympathize with y’all, that expresses very strong opinions about how those guys over there get sweetheart insider club review and funding privilege. While they themselves, of course, are subject to true and competitive merit-forward review.
In general, the in-house reviews (standing IC study section or IC SEP) are more likely to be accused of inferiority, lack of true competition and a smooth path for the insider club.
Within the CSR panels, well, there is some of this kind of grumbling based on the degree of IC capture that is represented with an added dose of complaining about “gatekeeping”.
I have colleagues on both sides of the divide. They are not shy when complaining about their latest grant disappointments.
The funny part is that it is all true. And it is all not true. In approximately equal measure.
Review is highly subjective. Bruce Reed of all people should understand this by now. The major driver is the attitude of the assigned reviewers and the panel at large regarding the importance / significance of the work. And people have a pronounced tendency to think the work they do is the most important. Talk to a NIAAA insider about how important it is to cure alcoholism and alcohol abuse. Or to a NIDA person who is, er, cocaine forward, about basic mechanisms of substance misuse and the way to make “real” progress. Or to a NIMH funded person about research funding for their favorite mental condition that doesn’t happen to involve substance abuse.
They may even quote you some epidemiological and funding stats that “prove” their position.
So how does Reed’s “appropriate competition” work in these more generalist* panels? As per always, our default hypothesis has to be one of percentages. A panel that is 80% “NIAAA folks” will give higher scores to alcohol proposals. A panel that is 80% “NIDA researchers” will score alcohol proposals lower.
How does this help respective Program officers? NIDA isn’t looking to fund any alcohol-focused work and NIAAA isn’t looking to fund a cocaine-only proposal. It is not necessarily “better competition” OR a more useful review outcome to pit these against each other all of the time. Who cares if the 1-5%ile is all NIDA proposals and the NIAAA ones score from 6-10%ile?
This is likely to occur, broad strokes. Statistically speaking. Particularly when one IC is smaller than the other, thereby generating fewer applications, thereby requiring, on the face of it, fewer peer reviewers on one of these newly generalist panels.
Don’t get too smug, NIDA-folk, NIMH is larger than us and the same dynamic could quickly put us on the back foot. And let’s not forget about Big Daddy NCI’s tobacco addiction portfolio.
The issues that are actually important to, e.g., NIAAA Program Staff for making funding decisions may not even be rigorously addressed, if the panel is basically responding “meh, it’s alcohol, pass“. Right? Sure, they can pull up anything they want from the ND-Competitive pool but these will not have the benefit of discussion or panel vote.
Look. I favor generalist study section panels that tend to get a diversity of IC assigned panels. I thought my formative years on BRLE were frikkin awesome in terms of the kind of cross-pollination of reviewers and proposals that occurred. The focus then was on behavior, regardless of the broad spectrum of disorders to which is was applied. But I always had the suspicion this only worked because there were more specialized / IC-captive CSR study sections as well as the in-house study sections.
Regardless, it is not trivially obvious that this more general competition will result in better review outcomes, in so far as the primary outcome is SUPPOSED TO BE assistance to the Program staff in deciding what to fund.
*Neuropathophysiology of Addictive Substances seemingly replaces NAL; Neural Basis of Motivated Behavior seemingly replaces NMB. Keep your eyes on them for rosters and, ultimately, funded grants to see how this re-organization plays out in practice.