This post intends to start a conversation about a single aspect of the merit review practices in DEB (itself just a narrow slice of NSF), the due date. Why the due date?  Well, for one, the implementation of a single deadline per year was one of the more prominent changes when DEB and IOS both released major revisions to their solicitations for FY 2012.  Plus, while the due date change was a major concern in initial reactions to DEB and IOS, and is something we continue to hear about, other programs at NSF are facing similar issues and have also looked at changes to the submission schedule (in BIO[1], and beyond).

None of the discussions we have seen online actually look at due dates across NSF which is a little bit disappointing because the variety of practices successfully employed across the foundation is awesome[2]. So, we took it on ourselves to remedy that.

Data Source (and DIY instructions)

The NSF.gov website hosts descriptions of every program and funding opportunity available Foundation-wide. The NSF funding opportunities page lists several hundred distinct funding opportunities, the vast majority of which are irrelevant to any specific individual (but there are probably twice as many as you realize that could support you). A list of these opportunities arranged by upcoming due date is available too, and was used for this analysis.  Since the funding opportunity list is constantly changing as information is updated repeating this exercise may yield slightly different results, we’ll give the standard caveat by noting that this analysis is based on the list as shown on 5 July, 2013.

The list of funding opportunities and due dates was sorted in a spreadsheet.  Duplicate programs (i.e., those listed twice because of two or more due dates in the next 12 months) were consolidated into single entries.  Opportunities not listed as receiving proposals in the next 12 months were excluded from analysis. The list was further modified by removal of supplement-only opportunities, and anything identified as a non-recurring opportunity- such as competitions to host major facilities.

For each of these funding opportunities, information was collected from the linked websites about where in NSF the program was hosted, the type of due date used, number of due dates per fiscal year, and spacing between subsequent due dates at which the same proposal could be resubmitted.

The final tally identified 217 current funding opportunities.

Recent Due Date Changes?

One of the things that becomes clear when reviewing the solicitations is that there has not been a major rush across the foundation to change the schedule of due dates.

Out of the 217 opportunities reviewed, only 10 appeared to have instituted a recent change (since DEB initiated pre-proposals) to their due date schedules.  Three of those are the core programs in DEB and IOS, along with the LTREB program, switching to once-a-year pre-proposals. The fourth is also in BIO: the core program in Molecular and Cellular Biosciences now also uses once-a-year deadlines, but without pre-proposals.  Another three programs in Engineering have switched from twice-a-year to once-a-year submission windows.  Biological Anthropology moved to an 8-month cycle (3 due dates per 2 years) and Geography and Spatial Sciences went to a single deadline with invitation-only resubmissions (called the “1+ schedule”).  Lastly, Mathematical Biology already had a once-a-year submission window, but shifted the dates of that window to facilitate co-review with programs in BIO.

By looking at the opportunity descriptions and checking archived versions of those with separate solicitations it becomes apparent that many opportunities follow due date schedules that go back 3, 4, 5, or more years.  Some current program descriptions for opportunities following the PAPPG (those without specialized submission instructions) date back to the late 1990s but changes to their schedules would be masked by the lack of separate solicitation documents.

While the challenges of maintaining a merit review system under increasing demand and supporting a robust science enterprise in uncertain fiscal times are concerns that many programs would probably identify with, the present variety and distribution of due date schedules across NSF funding opportunities is not the product of sweeping changes happening now or over the past several years.

Interpreting Due Date Patterns

There are four types of due dates currently employed across NSF:

• Deadlines- hard cut off dates for submission to an opportunity; requires a specific waiver or extension to get around in extenuating circumstances like natural disasters
• Target Dates- soft cut off dates; late submissions may get reviewed with on-time proposals or may be held by the program until the next cycle of review
• Submission Windows- ranges from ~2 weeks to ~2 months for submission; combines a deadline with a restriction on early submission.
• No Due Date- Rolling submissions; proposals are reviewed as received which either requires ad hoc (individual reviews) review only for timely turnaround (and first-come, first-served funding) or variable decision times to collect enough submissions for a review panel.

The distribution of current funding opportunities among the types of due date is summarized here:

Type	Count of Opportunities
Deadline	77
Submission Window	74
Target Date	49
No Due Date	17

Most funding opportunities[3], slightly more than 2/3 in the current list, employ a hard cut-off date for submission. Deadlines are widespread across NSF directorates.  Submission windows are mainly found in the Math and Physical Science and Engineering directorates. Almost all programs with target dates are limited to three directorates: MPS, SBE, and GEO. Most of the No Due Date programs are housed in GEO as well in the Atmospheric and Geospace Sciences Division.

For the programs employing some form of specific due date, we can identify the frequency of recurrence.  In the opportunities examined here, the least frequent due date came around once every other fiscal year and the most frequent happened twice every fiscal year.  By far, the most common schedule across NSF is a once-a-year due date:

Due Date Frequency	Count of Opportunities
Once or Fewer/Year	148
>Once/Year	52

But, those ~25% of opportunities with more than a single due date per fiscal year need to be looked at more closely.

In developing the changes in IOS and DEB, one of the problems identified in the past practices was that the 6-month cycle of deadlines conflicted with what is basically a 6-month or longer turnaround time for a review[4] and did not provide sufficient opportunity for PIs to receive reviewer feedback and effectively incorporate it into a resubmission for the next deadline.  On the other side, one of the concerns we hear is that the single deadline prevents timely resubmissions in such a way that disadvantages progress in BIO compared to other fields which have more frequent due dates.  We’ve already established that a single due date is actually the most likely schedule across NSF so that argument is dubious, but, it’s still worthwhile to drill down and look at how current opportunities with more than one due date organize them.

Timing Between Subsequent Due Dates for NSF Funding Opportunities with More than 1 Due Date Per Year
	4 mos. then                8 mos.	5 mos. then                      7 mos.	6 mos.	8 mos.
Count of Opportunities	22	11	18	1

What this shows is that even for those funding opportunities currently supporting more than one due date per fiscal year most are organized such that immediate resubmission cannot take place.  Given the time to coordinate reviewers and process decisions for a large number of submissions the opportunities employing 4/8 and 5/7 due date cycles effectively block immediate resubmissions.  Sure, one could submit to the first due date and then immediately resubmit 7 or 8 months later with perhaps a month to incorporate changes, but the reviews on that second submission would not be back in time for the next date 5 or 4 months after that.

In practical terms, even if your schedule is open enough to submit at both due dates, resubmissions are eventually pushed to a 12-month cycle.  And, as discussed in a prior post, a large portion of those submitters who do get funded on that immediate resubmission would have gotten funded on the first submission except for divvying up the budget between two due dates.

As an important aside, note that our preliminary proposal schedule follows a 5/7 split with 7 months between preliminary and full proposals.  This was deliberate because we believed that the smaller number of full proposals could be processed more quickly than previously and that the invited PIs needed whatever time we could save to write their full proposals (the time savings include tightening the full proposal review schedule, cutting out roughly a month of soliciting ad hocs for preliminary proposals, and notifying invitees before processing the large volume of non-invites: gaining about 10 weeks between notification and full proposal submission).

So, there are few opportunities at NSF where PIs are really able to effectively cycle any particular proposal to the same program more than once per year.

Lastly, we can look at which directorates house the six month interval funding opportunities.

Distribution of 6 mos. due dates by Directorate
Directorate	Count
GEO	12
MPS	1
Crosscutting	1
SBE	4

The programs with 6-month intervals are mostly in GEO and all of those are in the ocean sciences and earth sciences divisions which have very strong ties to DEB communities. Several of the GEO programs share a large disciplinary boundary with DEB and are frequent co-reviewers with all four core programs in DEB.  However, co-review is not dependent on matching schedules: we have extensive experience co-reviewing with other programs with single due dates, multiple due dates, and offset review cycles for pretty much everything outside of GEO.

What it all boils down to is this: The large majority of NSF opportunities are formulated in ways that allow a specific proposal to be reviewed only once per year. There are good reasons to have more than a single deadline that have to do with enabling over-committed PIs some flexibility in getting their ideas in. It’s something we’ve heard from you and are pondering, but the timing and organization of doing that raises many complications with the schedule of other opportunities that also need to be managed in DEB.