1. Why are some researchers unhappy with BI?

One of the primary findings of the Broader Impacts Toolbox Workshop was that community attitudes and outcomes regarding the broader-impacts criterion could be improved significantly by emphasizing to researchers the motivation for implementing the broader-impacts criterion, and improving applicant, reviewer and program officer understanding of the criterion to produce more consistent application.

A primary concern of researchers is that the lack of clarity regarding broader impacts leads to inconsistent application of the criteria in the merit review process. This feeling will only get stronger the more competitive funding becomes.

2. Why can't NSF be more clear about BI?

To be fair, there are a number of efforts at NSF to help researchers understand what NSF wants for BI. There are, however, a couple cultural problems.

• NSF is comfortable with the model they use to administrate and fund research, in which the funding agency responds to the community's submissions. NSF is not a prescriptive agency. They don't like telling researchers what to do, and they've continued that philosophy with the BI Criterion. While this leaves you a broad range of things to do, it's not always clear exactly what you should be doing.
• Researchers are selected for their expertise in research. Few researchers have experience with BI-type activities. Reviewers don't have the same motivation (wanting funding) that applicants have, so they may not spend much time trying to figure out the BI Criterion.

3. Does my proposal have to have Broader Impacts?

Yes. NSF can return proposals that do not address BI in the project summary and the proposal without review.

4. Are Broader Impacts weighed equally with Intellectual Merit?

No. Broader impacts are given different weights in different divisions and by different reviewers. There is no universal formula for determining how important BI is in the final decision wether to fund a program or not.

5. Can I hire someone to do my Broader Impacts?

There's a fine line between leveraging expertise and trying to get out of doing something yourself. Most reviewers can tell the difference. The Broader Impacts are supposed to be the Broader Impacts of the proposed research, so you and/or members of your research group need to have some intellectual contribution. If you can leverage an existing program that already has infrastructure -- connections with schools, for example, you may have more impact than if all your time is spent trying to find teachers to work with.

6. What makes a good Broader Impact?

One of the problems with BI is that some reviewers aren't quite sure what to do with it. Many know when they don't like something, or when a project is obviously contrived or doomed to failure; however, identifying what is good is a little more subjective. The best suggestions culled from the Broader Impacts Workshop are

1. The project has to fit with the research. Proposing something totally separate from the research doesn't make sense
2. Take advantage of your local strengths. If you have a long-running program of science lectures for the public, see if you can get involved in that instead of starting your own program -- just make sure you make it clear that you are contributing something, not freeloading
3. If you're going to propose something for the classroom, be aware of the literature in research on teaching and learning. We know a lot more about what works and what doesn't than we used to
4. Don't make a laundry list of activities trying to impress people with how much you're doing. The list of activities NSF put out is not a menu where you choose one from each column. Quality over quantity
5. Reviewers more and more want to see that you're making an effort to determine whether what you are doing works. You need to have some way to assess your activity and determine whether it is making an impact. (You ought to want to know this anyway so you don't waste your time doing something that is useless.)
6. Have a goal -- state clearly what you want to happen as a result of the activity (besides getting your research funding)
7. Reviewers are consistently inconsistent. Even if you follow all these suggestions, you still are likely to have someone criticize your activity. NSF badly needs to give reviewers more direction on whether they expect BI to be innovative, or whether implementing something we know works is sufficient.

7. Do any other funding agencies require BI?

NASA has a similar EPO (Education and Public Outreach) requirement and that community has been very active in developing resources to assist researchers in meeting those criteria. NASA utilizes a more well-defined model, in which outreach 'brokers' take the primary responsibility for EPO and a specific part of the budget is devoted to the activity. An excellent guide is available from the NASA Office of Space Science that provides lots of information about types of outreach, magnitude of effort and evaluation.

The National Institutes of Health have a quasi BI in the evaluation they call 'societal impact'. NIH has an advantage over NSF because everyone in government knows someone who suffered from cancer or heart disease. There's no question that curing these things is a good idea.