Dynamic Ecology

Welcome to our latest author interview! Recently I interviewed Scott Peacor on Peacor et al. (2025 Ecology), “Ecological meta-analyses often produce unwarranted results”. It shows that ecological meta-analyses frequently fail to correct for non-independence of effect sizes reported in the same paper, and that this failure has serious consequences. And it shows this is an entertaining way. I asked Scott about how a non-statistician ended up writing a statistical methods paper, the biggest problems with ecological meta-analyses, the use of rhetoric in scientific papers, and more. Come for the discussion of methodological issues in meta-analyses, stay for the advice (?) on how to wear down your collaborators, what scientific writers can learn from advertisers, and more.

The interview was conducted over Zoom. Both the questions and answers have been edited for clarity and brevity.

Jeremy: What’s the backstory of that Peacor et al. (2025)? How did that paper come about?

Scott: Yeah, I’m not a statistician. I did poke around and unfortunately never published a paper on my consternation with how we are doing ANOVA. Ecologists were not paying attention to what the underlying biological model is and how the model determines whether you have an interaction term or not. And then when I saw meta-analysis coming out—this was decades ago now—I would say I got kind of irritated. It seemed like what was coming out of them was way too much confidence relative to what was going into them. And, you know, I’m not a statistician, I’m not an expert in hierarchical models, but if somebody tells me a Little League team just beat a major pro team, even if I don’t know the rules of baseball, I can probably figure out something’s going on. And it really seemed like something was going on here. I thought this was going to be something that concerned everybody when they saw these papers. But that wasn’t the case. These papers have been cited thousands of times, and there has not been concern from the community.

I wasn’t interested in sussing it out myself, but nobody else was doing it. And so I decided to work on it. It took me a really long time, decades. And again, since I’m not a hierarchical statistics expert, and I like to work with others, I convinced Jim Bentz, who is an expert hierarchical modeler, to work with me. That was difficult. He was not interested. But what I would do is I’d put a paper on his desk and I knew he would look at it and he would get upset and he’d say, but Scott, I still don’t want to work on it. But after 3 or 4 meta-analysis papers on his desk, I wore him down. And I’m friends with Craig Osenberg, and Jim actually was from grad school too.

And then Jeremy, it really was eye-opening in trying to get funding. We wanted a grad student or postdoc to work on this, not us. And after the first rejection from NSF, I looked at DEB the last 5 years, and at the time I didn’t see any grants that were on how do we do science, how do we do experiments, and so on. Meta-analysis was being used so much, and we were saying, hey, there are some problems with the way we’re doing meta-analysis, let’s look at this. And NSF were not interested because it’s not novel. We tried to address that a second time, didn’t work again. And we were persistent. I didn’t give up. And the third time they gave us half what we were asking for.

I want my work to have a positive influence, not just have fun. This is not fun for me. I don’t really like statistics and I don’t like criticizing people. I don’t get a kick out of it. And so designing a statistic project to show that we ecologists are doing something wrong is not my idea of fun.

Jeremy: You said that the genesis of this was decades ago. Is it your sense that the statistical practice around meta-analysis have changed between the time when you started thinking about this and the time when the paper came out? Because like you, I’ve looked at a lot of ecological meta-analyses. I haven’t done a count, but my offhand impression is that a fair number of them these days are doing what you recommend and using hierarchical models to control for sources of non-independence. And that is a change. That would not have been the case back in the early ‘90s when meta-analysis first took off in ecology.

Scott: I’ll answer that question in two ways. Have things gotten better? In one way, I think they have gotten better. We now have easily accessible software to do hierarchical meta-analyses. So in that way we’ve gotten better. Have we become more reflective as a group of ecologists? I don’t think so. I think that MetaWin came out and we as ecologists know better than anybody what the problem with non-independence is and yet we had papers coming out using MetaWin that had 50 data points from the same paper that all looked exactly the same for the most part relative to the other papers, and we had ecologists that are completely willing to use that package knowing the problem with non-independence. Now, of course, you can’t expect a field ecologist to be at the level of a statistician, but you can expect them to have a statistician on their paper if they’re using a complicated tool that hasn’t been fully sussed out.

Jeremy: I wanted to ask you a little bit about the rhetoric in this paper, which I found entertaining. Because you do some analyses where you ask, “How often will we find a statistically significant effect of a moderator variable that truly has no effect, if we fail to correct for non-independence of effect sizes reported in the same paper?” But rather than just randomly generating a moderator variable that’s unassociated with the dependent variable, which I think is what most people would’ve done, you use moderator variables like, “Did the author’s last name contain an odd or even number of vowels?” It reminded me a bit of Fourcade et al. 2018, which you cite, which generated nonsense environmental variables for species distribution models by superimposing Old Master paintings on maps of Europe. So tell me a little bit about the thought process behind your use of nonsense moderator variables. Was it a deliberate attempt to grab the reader’s attention?

Scott: I’m certainly not trying to talk down to the reader or anything like that. It’s a matter of experience. When we were starting to study this, I read this book called Made to Stick. It’s about advertising. It turns out that just changing one word or something in an ad could sell 100% more of the product. I would talk with people about this work and I would say, “Hey, we randomised this and 40% of the randomised combinations came out significant.” Oh, yeah, they’d say, that’s interesting. But if I said, “Hey, I split them up by red state and blue state,” they’d say, “What?! What?! That was significant?!” [laughs] That’s just how our brains work. It was from experience talking about this that I realised, it gets the brain thinking more about how this just doesn’t make sense that these nonsense variables are having an effect. What was interesting with blue versus red states is I actually had people saying, “Well, in red states, you know, you’re in the interior of the continent, blue states are more along the coast. Did you …” They were trying to find an explanation for it, you know? [laughs] And so I had to have some moderator variables that readers would completely not be able to come up with an explanation for. “People with odd numbers of vowels in their last name, you know, maybe tend to work in forests rather than grasslands…” [laughs]

Some of my co-authors weren’t as enthusiastic about those nonsense variables. But one of our reviewers on our first submission really was very positive about it. That was encouraging, it meant that I wasn’t trying to do something nefarious or whatever. And yeah, figure 1 or 2, I forget, the one with the nonsense variables, is one of my favorite figures I’ve ever published. But it’s completely unneeded for the science, it’s just for fun, for salesmanship.

Jeremy: Here’s a nitty-gritty technical follow-up. For the datasets you’re looking at in this paper, using a hierarchical mixed effects model that accounts for non-independence of effect sizes from the same paper greatly reduces the Type I error rate, but doesn’t entirely reduce it to the intended level of 5%. You end up with a Type I error rate around 13%, still solidly above 5%. How much should we worry about that 13% and what, if anything, should we do about it? Could you try to take some still more sophisticated statistical approach that would somehow get that Type I error rate down to the desired level of 5%? Or should authors and readers just kind of informally apply a mental discount factor? Just keep in mind that the Type I error rate is probably closer to 0.1 or 0.15 than to 0.05, due to non-independence that can’t be easily accounted for statistically.

Scott: I certainly think we should worry about it. We talk about in the paper about a number of reasons for this 13%, which is a rough number. I want to also keep in mind that this 13% is conservative. If there are other sources of non-independence that are not captured in that 13%, it would be larger than that. So I would say, yeah, if you’re going to just use standard meta-analysis techniques and fit a model that includes a random effect for paper, maybe assume alpha is 0.2 or something.

Jeremy: In the discussion section of your paper, you raise the possibility of other sources of non-independence besides just non-independence of effect sizes reported in the same paper. You mention, for instance, the possibility of non-independence of effect sizes from different papers from the same lab group…

Scott: Earl Warner, Rick Relyea, myself, we do experiments pretty much the same way, you know? [laughs]

Jeremy: Okay, so there’s a good example. Anyway, I’ve been thinking about this, because we do already have a quite dramatic example of that sort of non-independence. It’s Clements et al 2022 Plos Biology, their meta-analysis of effects of ocean acidification of fish behaviour. They find that studies from one particular lab group, the Munday lab, really stand out as having very different results from others. And so that makes me wonder if there are any other examples–hopefully examples that don’t involve data fabrication as in the papers from the Munday lab! But I dunno, I worry a little that we’d really be opening a can of worms if we start expecting meta-analysts to routinely test for some lab groups consistently obtaining different results than others.

Scott: This problem you’re mentioning, I do think it’s likely a big problem in meta-analysis and we should look at it.

Jeremy: Do ecological meta-analyses have too much heterogeneity? Meta-analysis started out in medicine, of course, and in the big Cochran compilation of medical meta-analyses, the median heterogeneity, I2, is only 22%, which in ecology would be absurdly low. In ecology, it’s usually over 90%–over 90% of the variance in effect size is due to heterogeneity, not sampling error. I’ve talked to ecological meta-analysts who are totally comfortable with high heterogeneity, who take the view that that’s why you use a hierarchical mixed effects model–to account for heterogeneity, however much of it there might be. And I’ve talked to others who take the opposite view, that if your heterogeneity is anywhere near 90% that means you’re comparing apples and oranges and bricks in the same meta-analysis. Where do you stand on this?

Scott: I think it’s a matter of what makes sense, scientifically. It just does not make sense to me to put, say, metrics of population growth rate and population size from different studies in the same meta-analysis looking at the effect of some factor.

Jeremy: I agree with you on that. My own view on that would be that, I don’t think you should mix, say, population growth rate and population size in the same meta-analysis. But that the reason you shouldn’t is to do with ecological interpretability and not because you’re gonna be jacking up heterogeneity. Because I think our I2 values in ecology are going to be very high, even if you do take a lot of care to only include in your meta-analysis studies that are scientifically comparable.

Ok, next question: speaking as someone who, like you, has reanalyzed lots of ecological meta-analyses, what tends to worry me most isn’t non-independence of effect sizes within papers, though I do worry about that. It’s just having so few effect sizes from so few papers in the first place. The median ecological meta-analysis only includes something like 60 effect sizes from about 22 papers or so. 25% of ecological meta-analyses have 10 papers or fewer in them. So for most research topics in ecology we just don’t have all that many papers to go on. And further, the 22 papers included in the median meta-analysis were published over a period of 20 years. So ecologists collectively publish about one paper per year on a typical research topic. That’s just not much, right? It’s not a recipe for rapid scientific progress on most of the topics that ecologists study. Is that a problem? And if so, what, if anything, could be done about it?

Scott: Yeah, I feel like you’re asking me if I was NSF director, where would I put funds? And you know, one of the beautiful things about ecology is we’re all studying very different systems, and one of the difficult things about ecology is we’re all studying very different systems. So I don’t know the answer to that. I guess my feeling is it would be good if we had a little bit more top-down approach and somehow created more uniform studies.

In my view, the biggest issue we have in meta-analysis is that we’re using effect size metrics that don’t match the biology. Craig Osenberg has been banging this drum for a long time. I am taking some credit for convincing him to have another go. And so we are writing another paper with examples showing that when you use a typical ecological metric like log ratio or Hedges’ d you get the wrong answer I hope people will have a look at that paper because I think it is really important to pick the right underlying model for your statistical test or your meta-analysis.

This week: Alejandro Arteaga profile, choose your own adventure replication metric, the first nature photographer, and more.

This week’s Science has a profile of prominent–and controversial–Ecuadoran herpetologist Alejandro Arteaga.

Evolutionary ecologist Shikhara Bhat maintains a useful list of readings that many readers of this blog might find interesting and useful. Covers everything from popular science books, to philosophy of science, to scientific biographies, to presentation skills, to grad school life.

The US National Academy of Sciences has elected new members, including several ecologists.

Statistician Daniel Lakens argues that the recent high profile SCORE replication project should’ve just picked one way to define and analyze “replication”, rather than presenting 13 different definitions of “replication” and 13 different associated analyses in the main text of the paper. Curious to hear your thoughts on this, because I feel like the issue here generalizes. Was the SCORE project’s approach a sensible response to the fact that “replication” doesn’t have a single agreed-upon meaning? Was it a display of admirable intellectual honesty and transparency, given that the 13 different analyses produced quite a wide range of results? Or does the SCORE project’s approach amount to an abdication of intellectual responsibility–trying to have their cake and eat it too by writing the scientific equivalent of a “choose your own adventure” story?

Sticking with the challenges of defining and measuring “replicability,” in a new preprint Devezer and Buzbas argue that standard experimental approaches can’t distinguish between a situation in which different experiments on the same topic are all equally replicable, and a situation in which they’re not.

The story of pioneering nature photographer Cherry Keaton, whose work inspired a young David Attenborough. This was fascinating, just amazing stuff.

The advent of LLMs appears to have led to the filing of way more “pro se” US federal lawsuits (meaning, suits filed without the involvement of a lawyer).

A fun–but more than merely casual–attempt to use Claude Opus 4.7 to estimate how much earlier a bunch of major inventions could have been invented. Some interesting patterns emerge. Most major inventions probably couldn’t have been invented all that much earlier than they were, and the gap in time between “could be invented” and “actually invented” appears to be narrowing. Many (not all) of the (rare) inventions that could’ve been invented in a useful form much earlier are medical inventions. See also this list of readings, analyzing the factors that may have prevented various major inventions from being invented earlier.

If you’re an academic, some details of the new academia-set tv show “Rooster” are likely to annoy you, but you might like the show anyway.

When I was a new grad student, I was feeling pretty dejected after receiving multiple rejections in short order.* In a conversation with my advisor, he told me something that I still think of often, and that I’ve said to countless others over the years: academia is like baseball – you’re doing well if you’re batting .300

For those who don’t follow baseball, batting .300 (with ‘.300’ pronounced ‘three hundred’) means that you get 3 hits for every 10 at bats.** In professional baseball, batting .300 is great. According to last year’s Major League Baseball stats, only 7 players achieved this for the year. But it’s also a bit surprising in some ways that the best hitters in the world fail to hit the ball most of the time. 

It’s a little hard to say what my long term ‘batting average’ would be in academia, but, if you consider things like grant proposals and manuscript submissions together, there are certainly a LOT of rejections. And that was my advisor’s point. You get to grad school by having been generally successful. You certainly didn’t succeed in only 30% of the classes you took, for example. But in grad school and beyond, the amount of ‘failures’ (or what feels like failure, at least) is much higher, and it can be hard to adjust to the very consistent rejections that come with being in academia.

A recent Nature editorial called for more conversations about failure in science, and there have been folks who’ve created ‘shadow CVs’, to try to highlight how common rejections and failure are. The shadow CVs I know about are fairly old at this point (e.g., Jacquelyn Gill and Jeremy’s are from 2012), but their message is at least as relevant now. As Jeremy put it: If you get rejected, you’re in good company. Or, as Andrew Hendry wrote, rejection is something that all scientists have to deal with. 

I was reminded of this last week when I was talking to someone about grad school admissions. The person was asking for advice about applying to grad schools, and one thing that I thought was important to note is that there will most likely be a lot of rejections. I think it’s important to note this because it can be easy to take this personally – as a sign that you aren’t good enough – when really it happens for all sorts of reasons, many of which have nothing to do with the applicant (including that someone’s lab is full and they aren’t taking on any students in a given admissions year.***)

I’ve now reached the stage of being a senior ecologist, and I am certainly better at dealing with the frequent rejections. They can still sting, but not nearly to the extent they did earlier in my career. Both my resistance and resilience have increased over time, partially, I think, because less of my scientific identity is tied up in each proposal or manuscript.  

When I wrote above that it’s a bit hard to say what my long term batting average is, I thought “well, I guess I could calculate it?” But I decided not to, because I think it’s helpful to not focus on that, and not to view manuscript acceptances and funded proposals as my measure of success. Instead, I’ve spent a lot of time trying to train my brain to focus on the effort I put in and enjoying the process, which are things I can control, rather than the external markers of success (manuscript acceptances, grants, etc.), which are only partially in my control. 

Lately, I’ve been reading The Way of Excellence by Brad Stulberg. In it, he writes:
“When you find yourself losing – learn from what happened if you can, but then forget about it and move forward. Throughout it all, keep showing up as best you can….The most realistic and effective approach for sustaining excellence involves viewing the work as an ongoing practice, measuring and judging your level of attention and effort, and letting progress be a by-product of that. Focus less on any single result and more on the trend line…Instead of relying solely on visible progress for motivation, you’ve got to find joy in the work itself and the community in which you do it.”

All of this doesn’t mean you should totally ignore rejections – as Stulberg notes, you should learn from rejection if you can. For example, if reviewers are giving you constructive feedback, view that as a gift and try to address as much of it as you can.**** But you also need to not get bogged down in the feelings of failure that can accompany rejection.

In short: Academia and science involve a lot of rejection. It helps to know how common it is so that we don’t think we’re struggling a lot more than other folks are when we get rejected. And, when it inevitably happens, learn from it what you can, then move on.

* One of the reviews of my NSF Graduate Research Fellowship proposal included the phrase “Success is doubtful”. 

** ‘At bats’ isn’t quite the same as ‘times appearing in the batter’s box’ because of the way walks and some other things are counted, but you can consider them the same for these purposes.

*** Ecology differs from many other disciplines in that it has a culture where usually folks enter grad school to work with a particular person. This makes it important to reach out to that person before applying, including to find out if they are taking on a student in the coming year.

**** More thoughts on responding to reviewers can be found here and here.

Tyler Cowen asks a good question about papers in microeconomics and macroeconomics, that generalizes to ecology papers:

Imagine adding [a] button, to either micro or macro papers “Please rerun these results using what the AI thinks might be five other different yet still plausible [statistical analyses].”

We’ve talked a lot in the past about robustness in statistical analyses–the extent to which different plausible statistical analyses of a given dataset support the same scientific conclusions. “Many analyst, one dataset” papers try to get at this empirically by giving the same scientific question and associated dataset to each of many different analysts, and then asking how similar or different their results are. One problem with such papers is that they’re a ton of work–lots of people have to commit lots of time to organizing and coducting the project. Another problem with such papers is that they only tell you about robustness, or the lack thereof, for the analyses associated with that specific dataset. Probably, you don’t really care about that specific dataset, you care about some other dataset.

That’s why, in some fields (economics is one), it’s standard for authors to include robustness checks in their papers. They report each of several different, plausible ways of doing the statistical analyses, and show that they all lead to the same broad conclusions. That’s fine, but there are various problems with that practice. Readers have to believe that the authors have chosen their robustness checks appropriately, rather than cherry-picking from the multiverse of possible analyses. Another problem is that the resulting papers are super-long and extremely boring to read, especially for those readers who don’t really care about the robustness checks (perhaps because they believe them to have been cherry picked).

All of which is why I think Tyler Cowen’s asking a very good question here. I do think we may well be at the point where robustness checks don’t need to be reported by authors, because any readers who want to see some can just ask AI to provide them. Based on my admittedly-anecdotal-but-fairly-substantial experience with Claude Sonnet 4.6 and Claude Opus 4.7, I’m pretty confident that you could give either one most ecology papers, along with the associated data, ask for the R or Python code to run (say) five different yet still plausible statistical analyses, and within seconds you’d almost always get back error-free code, that does in fact run five different plausible statistical analyses, accompanied by crisp, informative, correct explanations of how each analysis works and how their results align or differ.*

Could you also use this approach to replace “many analyst, one dataset” papers? That is, rather than going to all the trouble of lining up a bunch of human analysts, would you get the same results if you just asked AI to pretend to be a bunch of different analysts? Somebody (maybe me!) should try it with the recent many analyst, one dataset paper by Gould et al.

*Hopefully unnecessary footnote: Do not jump to any conclusions from this post about my broad views on AI. This post merely points out one specific task that I have good reason to think that existing AI (specifically, the latest versions of Claude) would be good at.

This week: 2026 ESA award winners, a retraction from Journal of Ecology, life lessons from mathematicians, the Amish, and Ethiopian distance runners, HyperCard vs. ChatGPT, game theory vs. airlines, tell me again what caused the Challenger disaster, the best version of the periodic table, and more.

Congratulations to the 2026 ESA award winners!

Here’s a retraction of a Journal of Ecology paper, with some prominent senior authors. The retraction was prompted by concerns raised by “third parties” regarding apparent anomalies in the data. I suspect (but don’t know) that at least one of the “third parties” is a pseuodonymous PubPeer commenter on this thread about the paper. The authors admitted to possible “unintentional errors” and agreed to the retraction. Apparently, the errors could not be fully explained or corrected because the hard drive on which the data were stored failed. I found that surprising given that the paper was only published a year ago; I had assumed that cloud storage backup was pretty standard these days?

There is also a PubPeer thread about another paper by one of the same senior authors. This thread identifies numerous duplications in the raw data associated with the paper. There is a response from the senior author.

Following on from the previous links: the same pseuodonymous PubPeer commenter whose comments may have led to the retraction linked to above has comments on a number of recent ecology papers by various authors, including some high profile papers. Many of the comments identify data duplications. I’ve only had a quick look so far, but on my quick look I’d say that the duplications all seem to be well-documented. They look to me like data duplications that the authors and journals concerned should investigate, and if necessary take appropriate action in response.

Pólya’s classic How to Solve It as a general theory of problem solving.

If you read that last link (it’s an easy read, but long), you’ll discover that it dovetails with this: the cost of no friction.

An attempt to rigorously estimate the benefits to the US economy of having trustworthy official government economic statistics. I’m obviously not qualified to evaluate this estimate, but FWIW the estimated benefit is huge, even though it only considers one of the many mechanisms by which having trustworthy official statistics translates into economic benefits. I wish we lived in a world in which nobody thought it was necessary to try to estimate this. I’d rather live in a world in which everybody just took it for granted that duh, of course it’s good to have trustworthy official statistics, because how the hell is the government supposed to function without them?

Lessons from the Ethiopian approach to elite distance running. This was very interesting, even though I’m not 100% convinced that there are generalizable lessons here.

I grew up not far from the big Amish community in Lancaster county, PA, so I read with great interest this review of the 2010 sociological study of the Amish, An Amish Paradox. You might think that you could roughly arrange the various Amish orders on a one-dimensional continuum from “more conservative” to “less conservative”–but you’d be wrong. Fascinating stuff, with broader lessons for those who aren’t Amish. Really, really good essay review.

People hate being forced to participate in markets that they previously could opt out of.

The game theory of why airlines are always going bankrupt. This was really interesting. Obviously I’m not qualified to judge it, but FWIW it seemed plausible to me. It’s somewhat reminiscent of “lattice effects” in the population dynamics of flour beetle populations (wonky population dynamics that arise because population abundances can only take on integer values, even though the attractor might include non-integer abundances).

Captain Kirk vs. accountability sinks.

On the difficulties of causal attribution for purposes of assigning responsibility or blame. For example, the question “Who or what caused the Challenger disaster?” isn’t entirely an empirical question, because the answer will be dictated in large part by the level of abstraction at which you choose to frame it. Depending on the level of abstraction at which you choose to frame the question, the answer might not leave you with anyone to hold responsible. That’s because you can’t really hold broad enabling circumstances or “the system” responsible, at least not in the same way that you can hold a person or even an organization responsible. You can’t make “the system” feel bad, or socially ostracize “the system,” or fire “the system,” or fine “the system,” or put “the system” in jail. Now I’m wondering about the possible connections to extreme weather event attribution–the practice of attributing at least partial responsibility for specific weather events (specific hurricanes, heat waves, floods, etc.) to climate change. Usually, we (or at least I) think of blaming “the system” for some bad event as a way of diffusing responsibility for the event. But it’s my (possibly-incorrect) impression that extreme weather event attribution is intended in almost the opposite way. By attributing responsibility for the event to climate change (i.e. the climate system), it’s intended to motivate action to curb climate change.

Somebody wrote a transformer neural network model in [checks notes] HyperCard and trained it on [checks notes again] a Mac SE/30. Yes really. Link goes to the GitHub page, you can download and run it yourself. This reminds of the guy who showed that PowerPoint was Turing-complete, and the people who built a computer within Minecraft that you can use to play Minecraft.

Wait, the psychologist who wrote The Body Keeps the Score first made his name by promoting recovered memory bullshit in the ’80s?!

Vague scientist. This is old, but new to me and pretty funny.

Chemistry news you can use.

I recently had a conversation with someone during which that person indicated an area in which they felt they were lacking knowledge. This sort of introspection is helpful — it’s definitely important to recognize gaps in our knowledge and think about things it would be helpful to learn. (I have an old post on the importance of knowing and recognizing the limits of one’s knowledge). At the same time, sometimes thinking about this sort of thing can leave us beating ourselves up a little for what we don’t know (especially if we think lots of other folks know it).

At some point during that conversation, I drew this:

(If you’d like more poorly drawn cartoons related to imposter syndrome, here’s an old post.)

I think I’ve had conversations along these lines the most related to oral exams. In my department, part of the oral exam focuses on the student’s dissertation proposal, and part spans broad knowledge in ecology and evolutionary biology. One of the things that always strikes me during these exams is how often I have no idea of the answer to a question asked by another committee member. (It’s also striking how often I only know the answer based on having taught intro bio for a long time.) 

I’m far enough along in my career to find this interesting rather than threatening, but earlier in my career this sort of thing definitely triggered imposter syndrome. But, dissertation committees are generally assembled to try to have complementary expertise – the whole point is that not everyone in the room knows the same things! The Venn diagram of expertise of committee members will vary, of course, but it’s often the case that the overlap of all committee members is quite small.

So, spend time thinking about the boundaries of your knowledge and where you want to expand those. But don’t beat yourself up for things other folks know that you don’t.

And, if someone admits they don’t know something, take Randall Munroe’s advice: https://imgs.xkcd.com/comics/ten_thousand.png

This week: Peter Raven passes away, overestimating (one source of) biodiversity loss, LLMs vs. peer review, zombie ideas are cringe, the Trump administration vs. science, proof that I’m old, and more.

Botanist Peter Raven passed away on April 25 at the age of 89. He did classic work on coevolution in collaboration with Paul Ehrlich. But probably his greatest contribution to the fields of ecology and evolution was via his long-time leadership of the Missouri Botanic Garden. Under Raven’s directorship from 1971-2010, the Missouri Botanic Garden grew from a modest local institution into perhaps the world’s greatest hub for botanical research, education, and horticultural display. Raven, who was born in China, also arguably did as much as any single person to foster research collaboration between China and the US. Among his many awards and honors, he was a MacArthur “genius” grant recipient and received the National Medal of Science from President Bill Clinton.

Writing in Nature, Martin et al. argue that a recent high-profile paper dramatically overestimated biodiversity loss attributable to “outsourced deforestation.” I’m not an expert on this topic, but FWIW I found Martin et al.’s argument very well-made and quite convincing. Unusually for a comment on a published paper, they also draw some broader conclusions. Those conclusions concern the conduct of global-scale analyses of previously-published ecological data (often remotely-sensed data), and they’re pretty spicy. Curious to hear comments from others who know more than me.

The Trump administration has eliminated or restricted over 100 US scientific advisory panels, including the National Science Board that oversees NSF.

Related to the previous link: Andrew Gelman and Dorothy Bishop comment on acting CDC director Jay Bhattacharya’s interview with economist Emily Oster.

Zombie ideas in science never die, but maybe they can become cringe?

Very interesting interview with Fields Medalist Terence Tao on how LLMs are changing the work of pure mathematicians. I do still occasionally stumble across people on social media or ecoevojobs.net making quite clear, explicit, blanket claims that LLMs are useless for all tasks, and that anyone who thinks otherwise is somehow deceived or delusional. I have no idea how common such people are in the population at large; probably not that common I guess? But still, I am morbidly curious to know what those people would say to literally Terence Tao.

Related to the previous link, here’s Jeff Ollerton’s little experiment asking ChatGPT to review one of his own papers. It was a high-quality review, that overlapped with the human-authored reviews about as much as you’d expect human-authored reviews of the same paper to overlap with one another. In the comments over there, Stephen Heard argues that this is interesting and all, but ultimately neither here nor there because ChatGPT isn’t conscious and doesn’t truly understand the paper it’s reviewing. I respectfully disagree with Stephen on this. I think that in this context, worrying about whether ChatGPT is “conscious,” or “understands” what it’s reading, is irrelevant. I just don’t see why it matters whether or not ChatGPT is “conscious” or “understands”. There may well be good arguments against using LLMs in peer review–it’s not something I’ve thought much about yet–but “LLMs aren’t conscious and don’t understand what they’re reading” isn’t one of them.

Related to the previous link, here are Judith Mank and DE guest poster Gina Baucom in conversation about the use of LLMs in scientific publishing. One small thing that caught my eye: at one point Gina Baucom suggests that LLMs will tend to provide “general, quite vague” reviews because they lack specialist scientific expertise. To which, boy I dunno. Maybe that was true a long time ago (as in, like, a couple of months ago), but just based on reading Jeff Ollerton’s post it sure doesn’t seem like it’s true any more. Now I’m curious to try Jeff’s experiment with one of my own papers and its human-authored reviews, just to get another data point on this. Maybe this one, since it’s an open access paper for which I hold the copyright and it got quite contrasting reviews from humans. I think I could use Claude to do it, since I have my Claude account set so that anything I upload or type won’t be used as training data.

And here’s Jessica Hullman writing in opposition to the long-term trend–which predates, but has been accelerated by, LLMs–of stripping all the context out of scientific papers.

Why no university will ever adopt a “start-up culture.”

This week in links that aren’t about science or academia so probably aren’t what you came here fore: Lots of interesting data here on long-term trends in parenting in the US and around the world. One bit that caught my eye was that fathers spend a lot more time parenting than they used to decades ago, but moms still do a lot more. One implication is that total time spent on parenting has gone way up.

Also this week in links that you probably didn’t come here for: dispelling myths about Australia’s policies on asylum. This was of interest to me as a purely descriptive account of how policy changed over time and the effects those changes had. Whether those policies and their effects were good or bad is a very important but different question, the importance of which is not downplayed by the existence of the linked post, or by my link to it. In general, I am not a fan of ignoring or denying the distinction between description and endorsement, not least because I doubt that either science or society would survive the complete collapse of that distinction. Here’s a related old guest post from Peter Adler defending scientists as honest brokers.

I’m old enough to remember this well.

I need more choices! (said this Phillies and Red Sox fan)

In Star Wars: A New Hope, Obi-Wan Kenobi tells Darth Vader “If you strike me down, I shall become more powerful than you can possibly imagine.” Vader strikes Kenobi down anyway, and Kenobi ascends to a higher plane of existence and becomes even more powerful through his spectral guidance of Luke Skywalker.

I’ve been wondering if the same is true of the field of evolutionary biology.

That is, I’ve been wondering if evolutionary biology as a field is dying, or if it’s ascended to a higher plane of existence and thereby become more powerful than ever before.

I’ve been wondering this because of conversations with evolutionary biologists. A little while back, a leading evolutionary biologist said to me flat-out “Evolutionary biology is dying.” Simplifying and caricaturing somewhat*, this evolutionary biologist’s view was the genomics had replaced evolutionary biology. Researchers who care about whole organisms and their phenotypes were in decline, replaced by researchers who only care about genomes. On this view, evolutionary biology, like Obi-Wan Kenobi, has been struck down.

A little while after that conversation, I had a conversation with another leading evolutionary biologist, who said to me flat-out “Evolutionary biology won.”* Again simplifying and caricaturing somewhat, this evolutionary biologist’s view was that evolutionary thinking had thoroughly permeated all of biology. Self-identified evolutionary biology might appear to be dead, but only because evolutionary biology is everywhere now. All good biology research these days is underpinned by, and shot through with, evolutionary biology, even if researchers don’t often describe their work as “evolutionary biology” or publish it in the journal Evolution. On this view, evolutionary biology, like Obi-Wan Kenobi, is everywhere now, undetectable but nonetheless hugely powerful because it guides everything that biologists do.

Ok, over to you: which of those two views is closer to the truth? Is evolutionary biology dying, or has it won? Or are those just two different ways of saying the same thing?

p.s. In the comments, I’d also be interested to talk about why anyone would care about the fate of evolutionary biology as a field. I mean, I care, and clearly lots of other people do too! But why? Why should individual researchers care what field they’re in? And why should individual researchers care if their field is thriving or dying? One answer is “they care because it’s much harder to do research–get funding, publish papers, attract grad students, etc.–if you’re not part of a recognized, thriving field.” But is that the only answer? I don’t think so, but rather than spell out my own thinking I want to hear what others think. Related: our old discussion of whether ‘synthesis ecology’ is a distinct scientific discipline.

*I’m reporting a remembered verbal conversation; I don’t recall all the details and nuances. Fortunately, this is just a blog post that’s meant to be a conversation starter; it’s not important that I recall all the details and nuances. One nice thing about having a long-running blog with a smart readership is that I can trust our readers to take the reported conversations in the spirit intended–as a jumping-off point for an interesting discussion.

Recently, I saw a post on LinkedIn in which the author, Kevin Sanders, noted that there is a tendency for people in leadership roles to immediately change course in response to a complaint. Doing so feels like responsive leadership, but actually means you spend a lot of time managing noise, which can lead to burnout (and not accomplishing one’s goals).

His post really resonated. I have not been a department chair, but I have had various leadership positions, and dealing with the criticism – sometimes flaming hot criticism from close colleagues – can be really hard and, as Sanders notes, it’s easy to exhaust one’s self in response. Something that has helped me deal with this is thinking about hot potatoes. 

As I shared in an earlier post, when that initial criticism arrives, it is a flaming hot potato. Immediately trying to change something in response to that criticism is trying to eat the flaming hot potato – you’ll get burned in the process. Instead, it helps to hold the potato for a bit or, better yet, to put it aside and allow it to cool down. As came up in the comments on my earlier post, most of what we deal with in academia is not urgent.* New policies, for example, can take a small eternity to work their way through. To mix my metaphors, if you try to reply to each critique that comes in, you’ll exhaust yourself playing whack-a-mole. 

(I bought myself this sticker from RedBubble and have it on my desk as a reminder that I don’t need to deal with hot potatoes immediately.)

While the potato cools, you can try to get a sense of what the general consensus is. An important thing to remember is that the volume of the messages you hear does not reflect how widely held they are. Sometimes it’s just one person who is very upset and very loud. People who are happy with things probably are mostly going about their lives, not telling you loudly that they really like the proposed change. This is also important because, as in the anecdote Sanders shared on LinkedIn, sometimes someone frames a concern as being broadly held when they are actually the only one who has that concern, something that Brene Brown refers to as the ‘invisible army’. While the potato cools, you can get a better sense of how many people are unhappy with something, and whether those concerns seem to warrant an adjustment.

This is still a work in progress for me, and I’m enjoying having very few hot potatoes launched my way while on sabbatical. But I also continue to try to think about strategies that help with dealing with these sorts of things as they (inevitably) come up. I’d love to hear strategies that you use! 

* I sometimes need to remind myself, when I’m tempted to check on something just before bed, that I’m not a transplant surgeon. If something is a true emergency, my lab folks know to call me. For this reason, I’ve made Slack a ‘hidden’ app on my phone so I’m less likely to mindlessly check it at times when I am trying not to be working.

This week: RIP Desmond Morris, optimal replication, authentic vs. ‘authentic’, extreme heat waves vs. GDP vs. pseudoreplication, and more

Zookeeper and zoologist Desmond Morris, author of the hugely popular 1967 book The Naked Ape, has passed away at the age of 98. Link goes to the NYTimes obituary.

Americans’ trust in scientists is now highly politically polarized. This seems very bad to me, but I have no idea what could be done about it.

FWIW (very little!), I share this particular worry about AI. As another example besides the ones listed in the last link, AI apparently is now capable of writing quite good popular history books without much (any?) human assistance. The books synthesize information from various sources, are very readable, totally accurate, and don’t plagiarize previous work in the conventional sense of copying or lightly paraphrasing previous work without attribution.

In contrast (and at the risk of just straight-up contradicting myself), I confess I’m not worried about this. I’m not worried because I just don’t see any sign that any of this is affecting any journal I would ever even glance at. Perhaps that just shows I’m selfish and myopic? Andrew Gelman also comments.

Tell me again what an “authentic” memoir is? Good post, although you don’t really need hypothetical AI-based examples to make the point, do you? Can’t you make the same point by pointing to the many, many real world examples of fabricated memoirs that were widely praised for their ‘authenticity’ before the fabrication was revealed? Also, now I’m wondering if there’s a connection between people’s struggles to tell which memoirs are ‘authentic’, and people’s struggles to tell how other species experience the world? I want to think more about this; maybe there are some interesting commonalities?

What’s the optimal rate at which published studies should replicate?

A cogent-seeming (at least to me) critique of recent high-profile work claiming that extreme heat waves produce detectable reductions in national GDP growth rate. Basically: those high-profile results apparently are a product of pseudoreplication.

Recently, I used data from ecoevojobs.net to study the geography of EEB faculty job applications in the US. In passing, I noted that all 3,409 TT EEB faculty positions advertised in the continental US + Alaska on ecoevojobs.net over the past 10 years came from just 491 colleges and universities. That got me wondering: which colleges and universities in the US have been advertising the most TT EEB jobs lately? Of course, the answer will surely be large institutions that enroll many students, so I also got to wondering which institutions have advertised the most TT EEB jobs lately relative to their enrollment.

To answer those questions, I had Claude 4.6 look up the most recent enrollment data (full-time equivalents [FTE]) for the 491 colleges and universities in the dataset. I tallied up the number of TT job ads listed on ecoevojobs.net from each institution over the last 10 years, and the ratio of ads:enrollment. I spot-checked Claude’s work and didn’t find any mistakes. Claude did the job much quicker than I could have and just as accurately, so it was very helpful.

Of course, the number of TT job ads listed on ecoevojobs.net is an imperfect index of the true number of EEB jobs, because ecoevojobs.net lists some jobs that frankly aren’t “EEB” even under the broadest definition of the field of EEB. Further, while ecoevojobs.net is pretty close to comprehensive for US TT jobs that truly fall within the field of EEB, it’s not even close to comprehensive for jobs that fall outside the field of EEB. So don’t take any of the results too seriously.

With those caveats out of the way, here are the US universities (outside of Hawaii) that have advertised the most TT EEB jobs over the past 10 years, as best one can tell from these admittedly imperfect data:

• Florida (62 ads)
• Cornell (50 ads)
• Georgia (48 ads)
• Michigan State (45 ads)
• LSU (42 ads)
• Utah State (40 ads)
• Auburn (36 ads)
• Colorado State (36 ads)
• North Carolina State (35 ads)
• Washington State (34 ads)

Yes, that’s right, UC Davis isn’t in the top 10 (it’s 14th, with 31 ads).

Scrolling down the list of institutions that have advertised the most TT EEB jobs on ecoevojobs.net over the last 10 years, I was surprised to see Cal Poly Humboldt in 22nd place (23 ads). Cal Poly Humboldt only has an enrollment of 6,373 FTE; it’s much smaller than any institution above it and those immediately below it. That prompted me to ask: which institutions have advertised for the most TT EEB jobs on ecoevojobs.net over the past 10 years, relative to their enrollment? Here they are, in descending order:

• Scripps Institute of Oceanography (5 ads, 493 FTE. This is a special case because Scripps isn’t a fully independent institution, it’s part of UC-San Diego. And it’s primarily a research institute, hence the low enrollment. So feel free to ignore Scripps if you prefer.)
• Sweet Briar College (5 ads, 643 FTE)
• Eckerd College (15 ads, 1935 FTE)
• St. Mary’s College of Maryland (11 ads, 1583 FTE)
• New College of Florida (4 ads, 617 FTE)
• Rhodes College (13 ads, 2050 FTE)
• Warren Wilson College (5 ads, 845 FTE)
• Kenyon College (8 ads, 1743 FTE)
• Ursinus College (7 ads, 1608 FTE)
• Colby College (9 ads, 2100 FTE)
• Coe College (5 ads, 1313 FTE)
• Cal Poly Humboldt (23 ads, 6373 FTE)

Looking at the list of institutions that have advertised a lot of TT EEB jobs relative to their size over the past 10 years, I don’t see anything obvious that most or all of them have in common, besides being small. In particular, the institutions on that last bulleted list aren’t all struggling institutions that have been advertising lots of TT jobs because their TT faculty keep leaving or their searches keep failing. It’s true that Sweet Briar nearly closed a decade ago, which I suppose might have something to do with the number of jobs it’s advertised over the past 10 years (or maybe not). And New College of Florida has been in turmoil recently, but that turmoil started years after 3 of the 4 ads ran, so New College isn’t on the list because of their recent turmoil. Further, multiple colleges on that last bulleted list are on the US News & World Report list of top 50 national small liberal arts colleges; they’re the opposite of struggling. Similarly, Cal Poly Humboldt is on the list in part because they’ve been expanding since 2022, when they changed to a polytechnic. 9 of the 23 TT ads there that were listed on ecoevojobs.net were advertised in 2022 or later. Finally, if you’re a small institution, the difference between you ending up on the list above or not could well be a single ad that happened to get listed on, or omitted from, ecoevojobs.net. So definitely don’t read anything into which colleges are on that last bulleted list.

The same goes for the institutions on the first list–don’t leap to any conclusions about why any particular institution is on the list, or assume that they’re all on the list for the same reasons. For instance, most of the TT job ads at University of Florida that have appeared on ecoevojobs.net over the past 10 years predate the changes to the Florida tenure system in 2022. So Florida’s not on top of that first list because they’ve advertised a ton of EEB jobs since 2022 in order to replace a whole bunch of EEB faculty who’ve left since 2022. If I had to guess, I’d guess that most of the institutions on that first list are on there simply because they’re big; they employ a lot of ecologists, evolutionary biologists, and other biologists.*

UPDATE: A commenter asked which big universities hire the fewest EEB faculty. Good question! It’s not one I can fully answer. My data don’t include any institutions that haven’t had any TT jobs listed on ecoevojobs.net over the last decade, and I’m too lazy to go look up all those institutions. So below is a list of the universities with at least 30,000 FTE, that have advertised at least one TT position on ecoevojobs.net over the last 10 years, that have advertised for the fewest TT EEB positions over the last 10 years relative to their size. They’re in ascending order of ratio of advertised positions:FTE.

• Georgia State (52,034 FTE, 3 ads)
• Texas – San Antonio (33,400 FTE, 2 ads)
• BYU (33,397 FTE, 2 ads)
• UNLV (31,895 FTE, 2 ads)
• Illinois (54,016 FTE, 7 ads)
• Texas – Austin (54,476 FTE, 7 ads)
• Temple (34,282 FTE, 5 ads)
• Texas – Arlington (41,554 FTE, 7 ads)
• Houston (44,827 FTE, 8 ads)
• North Carolina (31,258 FTE, 6 ads)
• The only other universities with >30,000 FTE that have had at least one, but <10, ads on ecoevojobs.net over the past 10 years are Missouri, San Jose State, Florida State, UC-San Diego, Boston University, Texas State, and Pittsburgh.

Surprised to see UT-Austin and UC-San Diego on that list. It’s mostly a list of R2 or low-end R1 universities that aren’t known for EEB research, or for biology research in general. Keep in mind that which unis show up on this list is likely due to the vagaries of which ads-that-aren’t-really-EEB happen to end up on ecoevojobs.net. /end update

In conclusion, I looked into this because I was curious if any interesting patterns would turn up. None did, which means this is a pretty boring post. Sorry!

*FWIW, I spoke to a friend who’s a senior prof at one of the big institutions near the top of that first bulleted list. The friend confirmed that there hasn’t been anything unusual going on there over the last 10 years that’s led to an unusually large amount of EEB hiring.