In order for me to have graduate students and post docs, and to pay some of my salary, I need to write proposals. These proposals are to the National Science Foundation (NSF) or to NASA or to the Air Force. Over the years, a lot of things have changed about writing proposals. I thought that it may be interesting to discuss some of these things.
In talking to older researchers in the community, they would talk of the good old days of “writing” proposals, when program managers would call you up and say “I have a ton of money I need to get rid of, can I send some your way?” Well, those days are long gone. I hope that this post doesn’t sound like this. It probably will. Ugh. Sorry.
The general ideal of a proposal is that you have an idea and you can’t really go too far down the road of this idea without getting money to invest some time into it. You need the money to pay for someone’s (student, post doc, research scientist, etc.) time to pursue the idea. The proposal states some sort of question and some sort of methodology to address the question. You submit this proposal to the funding agency, they send it out to be reviewed by a few people, then it is often evaluated with a bunch of other proposals in a panel, and a decision is made on whether to fund it or not. The people who evaluate the proposal are similar to me – people who are knowledgeable about the topic and extremely overworked and have no time to evaluate a bunch of proposals. The people are the panel are asked to not only read over 4-6 proposals (being the main evaluator for 2-3), but be on a meeting with ~10 other people for a full week, arguing about which proposals should be funded. This is a thankless job, that, in truth, sucks as bad as it sounds.
Some rules of proposal writing that I have made up:
- Don’t piss off the reviewer. Whatever you do – do not piss off the reviewer. This means a LOT of things: (a) don’t trash talk other people’s models, work, instruments, etc.; (b) pay homage to other people’s work – cite people who are competitors of yours and talk about how it has advanced the field; and (c) don’t make reviewers work to review – make fonts easy to read, make figures easy to read, make tables easy to read.
- Be very clear. Don’t confuse the reviewer. Don’t use a lot of complicated math and terms that will confuse people. (Although, I feel like this last bit deserves a special discussion…)
- Make the proposal as short as humanly possible, while still being clear and complete. Don’t overwhelm the reviewer with a ton of details that don’t really matter. Only include the details that matter (ok, this is a tricky bit.)
When I started writing proposals, the main guideline was page count – proposals were limited to 15 pages. So, I wrote my proposals in two-column format with 11-point Times font. This, in my opinion made beautiful looking proposals (you can argue with Times, but it is a classic). Figures were either 1-column vertically-oriented, or 2-column horizontally-oriented. These proposals just looked nice. (I recognize in a digital age when people are reading on a computer, 2-column format it no longer preferred. Ah, the golden age… Shit…. Sorry…)
Since that time, both NSF and NASA have started to specify exactly what you are allowed to do: 12-point font, only certain fonts allowed, single column only, less than 55 lines of text per page, and 1 inch margins all around. Indeed, NSF has become more and more specific about everything – including your CV (i.e., your resume). In an NSF proposal, they tell you exactly what has to be included, to the point of saying that you can only list 5 “synergistic activities” (such as serving on committees for different community things), and that your papers that you produce are “Products” and you are only allowed to list 10, and they have to be labeled “Most Relevant Products” and “Most Recent Products”. If they are not labeled as such, your proposal can be returned without review. NSF also changes the expectations on the CV randomly every couple of years. Just to keep us on our toes? I don’t know why.
All government agencies have to deal with “conflicts of interest” when evaluating your proposal. This makes perfect sense – you don’t want someone who has a direct tie to me to review my proposal. But, the government agencies can define this in radically different ways. NASA defines a conflict of interest as anyone who has been paid by the place where you work any time in the last year. So, in my case – no one who has worked for UM in the last year can touch my proposal (reviewers or employees at NASA). At NSF, they make you fill out a spreadsheet and list anyone who you have, over the past 48 months: (1) been a co-author with; (2) been on a proposal with; or (some people include) (3) been on a talk with. This is a LOT of people. And, you have to do this with all co-Investigators on the proposal. So, there are a HUGE swath of people who are not allowed to touch your proposal.
Further, now a lot of proposals have mandatory sections, such as “Facilities and Equipment”, “Mentoring Plans”, “Data Handling”, etc. These are often ill defined, and it is unclear if reviewers actually read them. But, it is (or could be) part of the evaluation.
Because of all of these additional requirements, places such as the University of Michigan now require that you finish the proposal at least 5 business days before the deadline, so they can review it. Further, the College requires at least 2 days, and the department requires at least 1 day. So, you have to finish the proposal at least 8 business days before the deadline. And you have more requirements.
Places like NASA release final Announcements of Opportunities (AOs) 90 days before the proposals are due. This has been the same for at least 25 years, if not longer. But, the mandatory requirements on the proposal have gone up, and the effective due date has moved back. This has caused a huge amount of stress on both researchers and staff members who help researchers put together proposals and fill out all of the paperwork that needs to be submitted to the university to allow the proposal to move forward. There is such a large push on the paperwork that the proposal itself (the science) becomes secondary in the eyes of everyone except for (a) the person submitting the proposal, and (b) the reviewer.
NASA, in recognition of this, decided to do something about it. As is typical with NASA, the idea was amazing.
They decided that they would have a two-step evaluation process: (1) write a 2-page proposal which will be evaluated by a group of people and they will make a decision and invite about half of them to continue; then (2) write a 15-page proposal with all of the paperwork required.
Since that time, this has been expanded to almost all NASA programs (in Heliophysics) and has morphed into: (1) write a 2-page proposal about 60 days before the main proposal is due, in which the title and team are locked in – they can not be changed for the next stage. This proposal is no longer evaluated, but is often required. Sometimes they “encourage ” or “discourage” submission of the next stage, but the main reviewers are not provided this information, so it has no bearing on the review of the second stage proposal. Submit a full proposal, as before.
Because of the way universities (and businesses?) work, maybe you can see that you have effectively DOUBLED the amount of work that has to be done. The university treats the first step as a full proposal, so all of the paperwork needs to be filled out for that. And, because the title and team are set in stone, the idea has to be fully fleshed out. So, we went from having 90 days (minus about 8 business days) to 30 days (still released 90 days before the full due date, with a step-one proposal due 30 days after release – and the university still needs 8 business days – so roughly 3 weeks) to come up with our ideas, build a team, and gather all of the paperwork and submit. No pressure.
To add more chaos to this, NASA doesn’t do this on the same days of the year, so at least you could plan ahead. Because of the fact that NASA is 100% dependent on congress for funding to the line item, they often don’t know how much money they will have for programs until well into the fiscal year (continuing resolutions, anyone?) So, it is not their fault, but when they get money, they dump these AOs on the street, and then that starts the chain rolling.
Sadly, NASA specifically does a lot of stuff with good intentions, but this ends up biting them in the Saturn-V because they fear litigation. Someone gets a great idea (2-step process to ease work on proposers!), which then gets eaten by lawyers and gets turned into the opposite of what they want for fear of litigation.
Interestingly, it is within the power of NASA’s program managers to do something about this. Here is an example. Let’s say that NASA offers a program that has enough funds for 2 proposals and 10 are submitted. The review panel comes in and evaluates the proposals and let’s say that 4 proposals are rated as must fund. Well, NASA can’t fund 4 proposals. A program manager would never let a panel do this – they would go over and over and over these 4 proposals until 2 were rated as must fund, with maybe a backup or 2. But, if the panel were insistent, NASA would then have to use “programmatic considerations”, which are very much up to the program manager. They may look at the PIs and see that PI-1 has $10M in funding and is overwhelmed, while PI-3 has almost no funds, but a good publication record and is competent. Funds could go to PI-3 instead of PI-1. (NSF does this a fair amount, while NASA almost never does this.)
So, the example idea. Let’s say that the program manager has funds for 2 proposals. Instead of asking the panel to pick out the 2 best (and I have not even started talking about what is “best” yet!!!) proposals, the program manager asks for the 5-6 proposals that are reasonable – the ideas are ok, the team is ok, the methodology is ok. No whack-o ideas. The PM gets 6 proposals from the panel. They have to select 2. So, maybe they have some “programmatic considerations” or something, and whittle it to 5. Those 5 go into a lottery and then proposals are selected at random.
This is a crazy idea. But, if you did this type of thing, you could reduce the pressure on the proposal writers. Instead of having a 15 page proposal, you could reduce it significantly, since you are really only looking to see if the idea is nutty or not. You could save a huge amount of time on proposal reviews and panel reviews. See these papers: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4959526/ and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6996170/.
We have not even started to discuss biases in proposal evaluations or determining whether some research is better than other research. Further, it should be noted that by law, you can not charge time to a government agency while writing a proposal to the government (unless the grant says that it is ok.) So, a LOT of research scientists are, by law, required to write proposals in their own “free time”, This is incredibly painful.
Our proposal system is broken. It does not encourage innovation by giving researchers space to come up with fantastical ideas, but keeps them constantly putting out paperwork fires that arise because of ever-tightening deadlines and ever-mounting requirements for proposals. People are being stretched too thin to effectively come up with paradigm-altering ideas. And, if they had these ideas, they would be lost in the shuffle of funding only the people with the best “data management” plans.
NASA and NSF – please be paradigm shifters and change the way funds are awarded. Be scientific about it. Experiment. Try something new! Be bold! Be transformative!