Discussion: Deadlines and Due Dates


This post intends to start a conversation about a single aspect of the merit review practices in DEB (itself just a narrow slice of NSF), the due date. Why the due date?  Well, for one, the implementation of a single deadline per year was one of the more prominent changes when DEB and IOS both released major revisions to their solicitations for FY 2012.  Plus, while the due date change was a major concern in initial reactions to DEB and IOS, and is something we continue to hear about, other programs at NSF are facing similar issues and have also looked at changes to the submission schedule (in BIO[1], and beyond).

None of the discussions we have seen online actually look at due dates across NSF which is a little bit disappointing because the variety of practices successfully employed across the foundation is awesome[2]. So, we took it on ourselves to remedy that.

Data Source (and DIY instructions)

The NSF.gov website hosts descriptions of every program and funding opportunity available Foundation-wide. The NSF funding opportunities page lists several hundred distinct funding opportunities, the vast majority of which are irrelevant to any specific individual (but there are probably twice as many as you realize that could support you). A list of these opportunities arranged by upcoming due date is available too, and was used for this analysis.  Since the funding opportunity list is constantly changing as information is updated repeating this exercise may yield slightly different results, we’ll give the standard caveat by noting that this analysis is based on the list as shown on 5 July, 2013.

The list of funding opportunities and due dates was sorted in a spreadsheet.  Duplicate programs (i.e., those listed twice because of two or more due dates in the next 12 months) were consolidated into single entries.  Opportunities not listed as receiving proposals in the next 12 months were excluded from analysis. The list was further modified by removal of supplement-only opportunities, and anything identified as a non-recurring opportunity- such as competitions to host major facilities.

For each of these funding opportunities, information was collected from the linked websites about where in NSF the program was hosted, the type of due date used, number of due dates per fiscal year, and spacing between subsequent due dates at which the same proposal could be resubmitted.

The final tally identified 217 current funding opportunities.

Recent Due Date Changes?

One of the things that becomes clear when reviewing the solicitations is that there has not been a major rush across the foundation to change the schedule of due dates.

Out of the 217 opportunities reviewed, only 10 appeared to have instituted a recent change (since DEB initiated pre-proposals) to their due date schedules.  Three of those are the core programs in DEB and IOS, along with the LTREB program, switching to once-a-year pre-proposals. The fourth is also in BIO: the core program in Molecular and Cellular Biosciences now also uses once-a-year deadlines, but without pre-proposals.  Another three programs in Engineering have switched from twice-a-year to once-a-year submission windows.  Biological Anthropology moved to an 8-month cycle (3 due dates per 2 years) and Geography and Spatial Sciences went to a single deadline with invitation-only resubmissions (called the “1+ schedule”).  Lastly, Mathematical Biology already had a once-a-year submission window, but shifted the dates of that window to facilitate co-review with programs in BIO.

By looking at the opportunity descriptions and checking archived versions of those with separate solicitations it becomes apparent that many opportunities follow due date schedules that go back 3, 4, 5, or more years.  Some current program descriptions for opportunities following the PAPPG (those without specialized submission instructions) date back to the late 1990s but changes to their schedules would be masked by the lack of separate solicitation documents.

While the challenges of maintaining a merit review system under increasing demand and supporting a robust science enterprise in uncertain fiscal times are concerns that many programs would probably identify with, the present variety and distribution of due date schedules across NSF funding opportunities is not the product of sweeping changes happening now or over the past several years.

Interpreting Due Date Patterns

There are four types of due dates currently employed across NSF:

  • Deadlines- hard cut off dates for submission to an opportunity; requires a specific waiver or extension to get around in extenuating circumstances like natural disasters
  • Target Dates- soft cut off dates; late submissions may get reviewed with on-time proposals or may be held by the program until the next cycle of review
  • Submission Windows- ranges from ~2 weeks to ~2 months for submission; combines a deadline with a restriction on early submission.
  • No Due Date- Rolling submissions; proposals are reviewed as received which either requires ad hoc (individual reviews) review only for timely turnaround (and first-come, first-served funding) or variable decision times to collect enough submissions for a review panel.

The distribution of current funding opportunities among the types of due date is summarized here:

Type

Count of Opportunities

Deadline

77

Submission Window

74

Target Date

49

No Due Date

17

Most funding opportunities[3], slightly more than 2/3 in the current list, employ a hard cut-off date for submission. Deadlines are widespread across NSF directorates.  Submission windows are mainly found in the Math and Physical Science and Engineering directorates. Almost all programs with target dates are limited to three directorates: MPS, SBE, and GEO. Most of the No Due Date programs are housed in GEO as well in the Atmospheric and Geospace Sciences Division.

For the programs employing some form of specific due date, we can identify the frequency of recurrence.  In the opportunities examined here, the least frequent due date came around once every other fiscal year and the most frequent happened twice every fiscal year.  By far, the most common schedule across NSF is a once-a-year due date:

Due Date Frequency

Count of Opportunities

Once or Fewer/Year

148

>Once/Year

52

But, those ~25% of opportunities with more than a single due date per fiscal year need to be looked at more closely.

In developing the changes in IOS and DEB, one of the problems identified in the past practices was that the 6-month cycle of deadlines conflicted with what is basically a 6-month or longer turnaround time for a review[4] and did not provide sufficient opportunity for PIs to receive reviewer feedback and effectively incorporate it into a resubmission for the next deadline.  On the other side, one of the concerns we hear is that the single deadline prevents timely resubmissions in such a way that disadvantages progress in BIO compared to other fields which have more frequent due dates.  We’ve already established that a single due date is actually the most likely schedule across NSF so that argument is dubious, but, it’s still worthwhile to drill down and look at how current opportunities with more than one due date organize them.

Timing Between Subsequent Due Dates for NSF Funding Opportunities with More than 1 Due Date Per Year

 

 

4 mos. then                8 mos.

5 mos. then                      7 mos.

6 mos.

8 mos.

Count of Opportunities

22

11

18

1

What this shows is that even for those funding opportunities currently supporting more than one due date per fiscal year most are organized such that immediate resubmission cannot take place.  Given the time to coordinate reviewers and process decisions for a large number of submissions the opportunities employing 4/8 and 5/7 due date cycles effectively block immediate resubmissions.  Sure, one could submit to the first due date and then immediately resubmit 7 or 8 months later with perhaps a month to incorporate changes, but the reviews on that second submission would not be back in time for the next date 5 or 4 months after that.

In practical terms, even if your schedule is open enough to submit at both due dates, resubmissions are eventually pushed to a 12-month cycle.  And, as discussed in a prior post, a large portion of those submitters who do get funded on that immediate resubmission would have gotten funded on the first submission except for divvying up the budget between two due dates.

As an important aside, note that our preliminary proposal schedule follows a 5/7 split with 7 months between preliminary and full proposals.  This was deliberate because we believed that the smaller number of full proposals could be processed more quickly than previously and that the invited PIs needed whatever time we could save to write their full proposals (the time savings include tightening the full proposal review schedule, cutting out roughly a month of soliciting ad hocs for preliminary proposals, and notifying invitees before processing the large volume of non-invites: gaining about 10 weeks between notification and full proposal submission).

So, there are few opportunities at NSF where PIs are really able to effectively cycle any particular proposal to the same program more than once per year.

Lastly, we can look at which directorates house the six month interval funding opportunities.

Distribution of 6 mos. due dates by Directorate

Directorate

Count

GEO

12

MPS

1

Crosscutting

1

SBE

4

The programs with 6-month intervals are mostly in GEO and all of those are in the ocean sciences and earth sciences divisions which have very strong ties to DEB communities. Several of the GEO programs share a large disciplinary boundary with DEB and are frequent co-reviewers with all four core programs in DEB.  However, co-review is not dependent on matching schedules: we have extensive experience co-reviewing with other programs with single due dates, multiple due dates, and offset review cycles for pretty much everything outside of GEO.

What it all boils down to is this: The large majority of NSF opportunities are formulated in ways that allow a specific proposal to be reviewed only once per year. There are good reasons to have more than a single deadline that have to do with enabling over-committed PIs some flexibility in getting their ideas in. It’s something we’ve heard from you and are pondering, but the timing and organization of doing that raises many complications with the schedule of other opportunities that also need to be managed in DEB.


[1] WARNING: Story at this link contains outdated information! See MCB website for current due dates.

[2] Simply describing the diverse forms of merit review practices could easily fill dozens of blog posts. But, most of the underlying detail is already written down and available: it’s called the Proposal and Award Policy and Procedures Guide. While we are not advocating that you commit it to memory (in fact, don’t try, it gets updated too frequently), it is a very good idea to bookmark it for reference and skim the table of contents to get an idea of the range of options that exist at the very heart of the NSF merit review system.

[3] Not all funding opportunities are the same size with respect to budget and proposal demand. Some fields are supported by many small narrowly focused programs, others by a few larger broad programs.  BIO falls on the big and broad end of this spectrum.

[4] The official goal is “Inform [70% of] applicants whether their proposals have been declined or recommended for funding within 182 days, or six months, of deadline, target, or receipt date, whichever is later.” See “Goal 9” in this part of the FY2014 budget request for details: http://www.nsf.gov/about/budget/fy2014/pdf/50_fy2014.pdf

One thought on “Discussion: Deadlines and Due Dates

  1. Pingback: Analysis of IOS preliminary proposal submission system, Part 1 of 3 | IOS in Focus

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s