Fiscal year 2017 (FY 2017) officially closed out on September 30. Now that we are past our Fall panels, we have a chance to look back and report on the DEB Core Program merit review and funding outcomes for FY 2017.
This post follows the format we’ve used in previous years. For a refresher, and lengthier discussions of the inner workings of the metrics, you can visit the FY 2016, FY 2015, FY 2014, and FY 2013 numbers.
FY 2017 Summary Numbers
The charts below all reflect DEB Core Program projects through each stage of the review process: preliminary proposals, full proposals, and awards.
DEB reviewed 1384 preliminary proposals received under the DEB Core Programs solicitation and LTREB solicitation in January 2017, about 28% of which were invited to the full proposal stage.
The preliminary proposal invitees were joined at the full proposal stage by 1) Direct submissions to DEB under the CAREER, OPUS, and RCN solicitations, and 2) Projects shared for co-review by another NSF program. Altogether, 514 full proposals were reviewed in DEB during the Fall of 2017 (this includes the OPUS, CAREER, RCN, and co-reviews).
From this pool of full proposals, DEB made awards to 119 projects. Below, we present and discuss the Division-wide success rate and some select project demographics. The demographic numbers are presented as proportions for comparison across the review stages.
Success Rate
Figure 1: DEB Core Program success rates from fiscal year 2007 through the present. Prior to fiscal year 2012, there were two rounds of full proposal competition per fiscal year. Preliminary proposals were first submitted in January 2012, initiating the 2-stage review process and leading to the fiscal year 2013 award cohort.
Calculation Notes:
Preliminary proposal success rate is calculated as the number of invitations made divided by the number of preliminary proposals submitted.
Full proposal success rate is calculated as the number of awards made, divided by the number of full proposals reviewed, including OPUS, CAREER, and RCNs.
Note that post-2012, under the preliminary proposal system, the set of full proposals reviewed is ~80% invited full proposals and ~20% CAREER, OPUS, RCN and co-reviewed proposals, the latter of which were exempt from the preliminary proposal stage.
Overall success rate is calculated as the number of awards made divided by the total number of distinct funding requests (i.e., the sum of preliminary proposals submitted plus the exempt CAREER, OPUS, RCN, and co-reviewed full proposals).
Reminder: Elevated success rates (in 2009 and 2012) were due to:
- a one-time ~50% increase in funding for FY2009 (the ARRA economic stimulus funding) without which success would have been ~13-15%; and,
- a halving of proposal submissions in FY2012 (the first preliminary proposal deadline replaced a second full proposal deadline for FY2012), without which success would have been ~8-9%.
Individual and Collaborative Projects
As a reminder to readers: the gap between the proportion of single investigator projects in the preliminary proposal and full proposal stages is due to the single-investigator proposals in the CAREER and OPUS categories. The CAREER and OPUS proposals are not subject to the preliminary proposals. . The absence of CAREER and OPUS proposals at the preliminary proposal stage lowers the single investigator proportion of the preliminary proposal counts relative to the historical full proposal baseline.
Figure 2: The proportion of DEB Core Program projects lead by a single PI over time and at the different stages of merit review.
The proportion of collaborative proposals in our award portfolio rebounded from last year’s drop and is near the all-time high for both full proposals and awards. This is consistent with the general trend toward greater collaboration over the past decade and beyond.
Figure 3: The proportion of DEB Core Program projects with two or more different institutional participants over time and at the different stages of merit review.
Readers may notice that the collaborative and single-investigator groupings don’t sum to 100%. The remainders are intra-institutional multi-PI arrangements; such projects are certainly intellectual collaborations, but they are not a “collaborative project” per the NSF PAPPG definition (Figure 3).
Early Career Scientists
The best identifier of researcher career stage is the difference between the year that the PI obtained their Ph.D. (as self-reported by the PI) and the submission date. This “Degree Age” metric can be used as a proxy for how long each individual has been in the population of potential PIs.
Figure 4: Distribution of degree ages among PIs on DEB Core Program full proposal submissions.
Figure 5: Full proposal success rates for PIs on DEB Core Program proposals by degree age. Figure displays annual data and a 5-year mean for the period of the preliminary proposal system in DEB.
Gender & Predominantly Undergraduate Institution (PUI) Status
Figure 6: The representation of female PIs and predominantly undergraduate institutions in DEB Core Program proposals and awards. These two groups were noted by the community as groups of concern that would be potentially impacted by the pre-proposal system.
Concluding Thoughts
This concludes our 5th fiscal year wrap-up. This series originally started in 2013 to track metrics some PIs thought would be sensitive to preliminary proposal implementation in 2012. However, given our move to a no-deadline model and the elimination of the preliminary proposal system, next year’s fiscal year wrap-up may look a little different. We still plan on reporting our funding rates but the other metrics will change.
DEBrief 