Comments at a recent panel brought to our attention[i] one nugget of reviewer wisdom that we were surprised to learn isn’t widely known but makes sense to share here with many of you taking part in NSF reviews. Since review assignments for the latest round of preliminary proposals are making their way into the community, we thought it timely to post a quick explanation of the less-advertised options for rating the proposals you’ve been asked to review.

When you review an NSF proposal, you don’t need to give it a single letter score of E (excellent), V (very good), G (good), F (fair), P (poor). In the reviewer system (through FastLane) you can check more than one box for “Overall Rating” to give a score between two of the ranks, like V/G or G/F when the 5 point system feels too coarse. Whether you check one box or two, however, the purpose is to capture a single “Overall Rating”. In other words, we ask reviewers to synthesize their evaluations of intellectual and broader impact merits into a single score. Therefore, if a reviewer provides a split score we (and the PI) view it as indicating a score that is in between the two categories. A split score that spans more than two adjacent ratings, or is meant to reflect different scores for different aspects of the proposal, is not especially useful since we don’t know how the reviewer rated the overall proposal on balance.

On the flip side, be careful if you’re trying to select a score and check the wrong box: FastLane doesn’t automatically clear the first choice when you make another selection, creating the potential for unintended scores like “V/G/F/P”.

Generally, the written content of the review matters more than the rating score: we don’t have an average-score-based “funding line”. Nonetheless, scores aren’t ignored: they’re a concise indicator of a reviewer’s opinion, can be really helpful for interpreting the written content. Scores are incredibly useful in managing panel discussion because they allow us to compare general opinions and quickly see if the reviewers are all starting from a similar place or whether there may be divergent views to work through. Being judicious in your assignment of scores can also be useful to you as a reviewer/panelist to differentiate between your many assignments and remember them through hours of discussions. On rare occasions, a reviewer may opt not to provide an overall rating at all and just provide the written comments. While acceptable, we discourage doing this on a regular basis and expect to see only a handful each year out of 10,000+ reviews.

Keep these tips in mind next time you’re reviewing for us.

~~mentally insert your favorite Public Service Announcement catchphrase and jingle here~~