With so many new proposal reviewers joining us, we thought now would be a good time to re-share an explanation of rating options on proposals you review.
Selecting an Overall Rating
When you review an NSF proposal, you don’t need to give it a single letter score of E (excellent), V (very good), G (good), F (fair), P (poor). In the reviewer system (through FastLane) you can check more than one box for “Overall Rating” to give a score between two of the ranks, like V/G or G/F when the 5 point system feels too coarse. Whether you check one box or two, however, the purpose is to capture a single “Overall Rating”. In other words, we ask reviewers to synthesize their evaluations of intellectual and broader impact merits into a single score. Therefore, if a reviewer provides a split score, we (and the PI) view it as indicating a score that is in between the two categories. A split score that spans more than two adjacent ratings or is meant to reflect different scores for different aspects of the proposal, is not especially useful since we don’t know how the reviewer rated the overall proposal on balance.
On rare occasions, a reviewer may opt not to provide an overall rating at all and just provide the written comments. While acceptable, we discourage doing this on a regular basis and expect to see only a handful each year out of 10,000+ reviews.
On the flip side, be careful if you’re trying to select a score and check the wrong box: FastLane doesn’t automatically clear the first choice when you make another selection, creating the potential for unintended scores like “V/G/F/P”.
How Overall Ratings are Used
Generally, the written content of the review matters more than the rating score: we don’t have an average-score-based “funding line.” Nonetheless, scores aren’t ignored; they’re a concise indicator of a reviewer’s opinion, and they can be really helpful for interpreting the written content.
Scores are incredibly useful in managing panel discussion because they allow us to compare general opinions and quickly see if the reviewers are all starting from a similar place or whether there may be divergent views to work through. Being judicious in your assignment of scores can also be useful to you as a reviewer/panelist to differentiate between your many assignments and remember them through hours of discussions.
Keep these tips in mind next time you’re reviewing for us.
If you’re having issues viewing scores there is a dedicated helpdesk for FastLane issues at 1-800-673-6188 and an extensive online help resource.
DEBrief 