Friday, August 19, 2011

Coburn Report on NSF


(Copied from a July post on my "home" blog)

Last week I responded to my Congressional delegation on Senator Tom Coburn’s report entitled ““The National Science Foundation: Under the Microscope” (http://coburn.senate.gov/public//index.cfm?a=Files.Serve&File_id=2dccf06d-65fe-4087-b58d-b43ff68987fa). I was aware that some scientists whose projects were represented in the report had already responded through blogs and other public forums. Having worked at NSF, however, there was an aspect of the report to which I could respond specifically, and I decided to stick pretty closely to observations that this unique perspective offered. My letter to Senator Lamar Alexander (R-TN) is below; the same letter was also sent to Senator Bob Corker (R-TN) and to Congressman Jim Cooper (D-TN) representing Tennessee’s 5th District.

*******

July 21, 2011
Senator Lamar Alexander
3322 West End Avenue, #120
Nashville, TN 37203

Dear Senator Alexander:

I recently read the entirety of Senator Coburn’s report “The National Science Foundation: Under the Microscope.” You are one of my senators and I am thus writing to you to express significant concerns with the report, focusing on those for which I have something of a unique perspective. From July 2007 through August 2010 I was on leave from Vanderbilt University, serving as a Program Director at the National Science Foundation (NSF) in the Directorate of Computer & Information Science & Engineering (CISE). It was a pleasure and honor to serve at NSF as a member of a hardworking, dedicated team. I regard my time at NSF as service to my country of which I am very proud.

Foremost, I worry that there seems to be an attitude of disrespect for NSF staff permeating the report. For example, the report includes a section “NSF Flying High with First-Class Junkets” (p. 14), with no indication in the section that anyone did this. Indeed, I never flew first class at NSF, or on a junket, nor do I know of colleagues who did. In a second example, when commenting on NSF’s desire to become environmentally friendlier, the report says “Some might find it interesting to note, then, that the NSF currently owns 375 vehicles, including 52 sports utility vehicles” (p. 15). This statement seems flippant and is vague, but is suggestive that NSF staff are hypocritical, not environmentally conscious and/or that the vehicles are not used for science. A third example is the report’s claim that porn surfing was “pervasive” (p. 15), with at least six citations of the same article in The Washington Times. This statement is very wrong. I can believe that such cases, though anomalous, have consumed a large part of the Inspector General’s time in the recent past, since a tiny proportion of a group is often responsible for a large proportion of the angst. We have seen a very recent example in Congress of misbehavior that consumed large amounts of time and energy, but I would not claim that such behavior was rampant in Congress.

My experience at NSF contrasts with the report’s representation of NSF staff activities. My colleagues and I performed many diverse tasks, including the vetting of research proposals with input from other scientific experts, preparing research solicitations, preparing and giving outreach talks to the public, and representing NSF and our country overseas. It is because of administrative and scientific staff dedication that the agency functions as well as it does in spite of very heavy workloads. I worked 60+ hour weeks and this really was pervasive across the Foundation. It appears that even when the President and Congress agree on budget increases for NSF, it is for scientific initiatives that come with yet more overhead and not for the addition of staff to deal with that overhead. I very much encourage you to consider additional funding for staff as well as scientific initiatives.

I believe that the report’s tone will cause many to reject the report entirely, to include points that I think have validity. For example, I generally agree with the report’s statements that a discussion about funding priorities is important. In fact, the very hard discussions I had with my fellow program directors at NSF were on what projects to fund given differing opinions on priorities and our limited budget. Our budget only allowed us to fund about one-third of the projects that had been judged by scientific experts to be most worthy of funding, which in turn was about one-third of all projects submitted; these proportions varied across the agency. The heartbreaking part of my job at NSF was that the majority of projects worthy of funding could not be funded, and my colleagues and I knew the costs for science research in our country and the costs to faculty members and students behind that research. NSF personnel take the job of assessing the scientific qualifications of projects and funding priorities very seriously.

I also agree in general with the report on the importance of metrics and tools to evaluate the payoffs of scientific investments; I believe that NSF staff would welcome such tools with open arms. Data analysis and visualization tools are critically needed to track scientific investments and to evaluate the US funding portfolio within and across agencies. The report highlighted the importance of the STAR Metric initiative, which is aligned with the Science of Science and Innovation Policy (SciSIP) program housed in the Social, Behavioral and Economics (SBE) Directorate of NSF. Thus, I was surprised that the report also recommended eliminating the SBE Directorate. I can’t emphasize enough how wrong I think this would be. I am a computer scientist and daily I witness how computing technology is transforming the ways that humans interact, perceive, decide and learn. The last thing we need is to cut research on understanding human behavior in a time of transformative technology. We should understand, for example, what video games and social networks are doing to our children and all citizens, so that we can design technology to enhance learning and decision-making, not diminish it.

Finally, I fear the report’s tone because it comes across as demeaning public servants whom I know to be dedicated, talented and industrious. I am not sure where degrading stereotypes of ‘government bureaucrats’ originated, but I for one, a lifelong academic, gained an incredible respect for the hard work and brains of federal staff, not just at NSF, but in agencies and departments across government.

Government colleagues can and should offer constructive criticism to one another, but I believe Senator Coburn’s report doesn’t paint an accurate picture of those at NSF who loyally and diligently serve their country. In particular, I wanted to convey my experience to you that NSF is an institution that Americans can be proud of, respected and emulated the world over, with staff who do their best in making difficult decisions on matters of national and scientific importance, despite a limited budget and a heavy workload.

Thank you for your attention and for your service to our country.

Respectfully,
Douglas H. Fisher

**** End letter ****

Generally, what significant empathy I have for the challenges currently faced by Congress and the President is due to my NSF service. It also makes me sick that so many members of our government appear to be so disrespectful of each other.

Despite my fears about the Coburn report, I think that some good things could come of it and the responses that are following. Notably, I hope that scientists, after reading the report, see the vital importance of communicating science to the public, to include Congress and scientists in disciplines other than their own. I’d like to see every research team be associated with those skilled in communicating scientific findings and their national and international relevance to the public. Universities have individuals skilled in communicating science to the public, but they are probably too few and far between – communication can be integral to research projects, and researchers can ask for the funds necessary to support that activity. Rather than blog posts erupting after reports such as this to explain and justify scientific research, maybe we’ll see more proactive outreach.

Generally, my experience suggests that scientists and engineers can more embrace their role as citizens, and all citizens should recognize that science and engineering are integral to citizenship. I could be wrong, because I might be working from a biased sample, but my sense is that not a lot of scientists write their elected officials, advocating more funding for science research and the like. I hope that scientific professional organizations not only respond themselves, which is happening, but that they encourage civic engagement by their individual members. Its probable that the climate scientists have internalized this message over the last few decades, and the social scientists may be getting it as well.

In any case, while my 3+ years at NSF reenergized the teacher in me and the researcher in me, it may have reenergized the citizen in me most of all. I hope so.

Virtual Panelists and Thoughts on Assessing Science


(Copied from a February post on my "home" blog -- its still timely)

I was quoted (correctly) in a commentary in Science entitled “Meeting for Peer Review at a Resort That’s Virtually Free” (http://www.sciencemag.org/content/331/6013/27.full) — nice. The article talks about the advantages of using virtual technology to convene scientific review panels at the National Science Foundation (NSF) and other Federal agencies like NIH instead of flying the panelists cross country for a physical meeting of a day or two. The article focuses on the very cool technology of virtual worlds, like Second Life (http://secondlife.com/), to host such activities, but video conferencing and phone are alternatives. Some might think that a phone is primitive technology; perhaps, but a land line is extremely reliable and not as primitive as an airplane, at least relative to the task of putting talking heads together.

In the Science article the lead reason presented for using virtual participation for NSF (NIH, etc) is that it saves money. I am somewhat conflicted on how to talk about this. On the one hand, the article says that approximately 19,000 reviewers were used by NSF last year; even if all of these had traveled to NSF, but were replaced by 19,000 virtual panelists, saving $1000 per panelist (all VERY optimistic), that would be savings of $19,000,000 (19 million); that may seem a lot, but only a dent in NSF’s 7 Billion dollar yearly budget, which in turn is only a wafer thin (0.0002) slice of the federal budget of 3.64 TRILLION dollars; http://nationalpriorities.org/en/resources/federal-budget-101/budget-briefs/federal-discretionary-and-mandatory-spending/). Nonetheless, the current budget debates suggest that President and Congress are focused on pocket change, so perhaps demonstrating any small budget cut would be the sacrificial lamb needed to buy NSF and science/engineering generally some political goodwill; and when such savings are spread across all such travel across all agencies, it could be significant.

I worry though that if monetary savings are presented as the lead story when talking to researchers/panelists, it will convey the (wrong) message that quality in the review process is being sacrificed to save money, and for the researcher/academic this could easily be a source of disappointment, if not resentment — after all, every two days the national debt grows by over 8 billion dollars (http://www.usdebtclock.org/), comfortably more than the yearly NSF budget! Amazing! Deeply discouraging. If virtual participation on panels picks up, it would be terrible to have scientists and engineers (or heck, anyone who cares about the US’s future) falsely believing that a few bucks are being saved out of the hide of science and engineering research in the US. Thus, I winced a bit that monetary savings took the lead in the article.

In my three years at NSF (2007-2010), fully 3/4 of my over 200 panelists were virtual. While I went in to NSF with an interest in virtual paneling to reduce ecological footprints, what pushed me over the edge (almost immediately) was that I couldn't find available rooms for panels on the days I wanted to hold them. I quickly saw more and more reasons for virtual paneling (as did others), and I recorded my experience on the NSF wiki. On my last active day as a program director, a Saturday (8/21/10), I mass emailed the entirety of NSF on the virtual paneling experience before leaving the office for the last time :-), and even on a 
weekend, received responses from interested program directors before my email address expired at midnight!

The reasons for using virtual panelists, and particularly in giving panelists a CHOICE on going to Washington DC (or anywhere, and for any organization) or participating virtually are many. The article does NOT address most of these reasons.

(1) Virtual participation reduces travel wear and tear on panelists — West coasters, rural and others underserved by airports, those overseas — this was alluded to in the article; travel is a great inconvenience/impossibility for many, but its a pleasure to many others, and/or an opportunity to network. I like a system in which panelists weigh the costs, benefits and choose for themselves on whether to attend physically or virtually. So, rather than suggesting an artificial dichotomy between all physical and all virtual, lets recognize that there are hybrids that allow both. Behavioral economics suggests that one can influence the proportions of the two kinds of participants by making one form (e.g., virtual) the default, and indicating the other option (e.g., physical) as welcome. If default specification strikes some as “mind control,” I suggest that its preferable to requiring one form (e.g., physical).

(2) Virtual participation broadens participation to many who might not otherwise serve — parents of young children, senior and very busy researchers, those who have to teach a class, and/or those who an agency might not otherwise ask because of monetary cost, such as those overseas, who may well have special expertise in an area that would benefit the US.

(3) Virtual participation reduces wear and tear on government agency staff; this reason probably would be the most under-appreciated by those outside of government agencies; I am talking here about administrative staff primarily — arranging catering, cleanup, travel and reimbursements, and many other misc responsibilities require a lot of time. Many federal admin staff commute an hour or more EACH way and have oodles of other responsibilities to which they must attend. Early in my 3-year NSF career I attended an outreach talk at San Jose State University, in which the NSF/OLPA presenter showed a graph of the number of proposal actions by NSF, and these actions were growing at what was clearly greater than a linear function over the last 25 years. In this same period staffing numbers remained flat. The implications for workload are obvious. I don’t see these trends changing — even when the President and Congress agree to funding increases, it is for new programs that come with additional overhead, and not for increased staff; the research community, given the historically very low funding rates, will continue to push proposal pressure up. Streamlining in operational efficiency is absolutely necessary. Using virtual participants offloads burden from staff, so that they can do other necessary things and maintain quality.

(4) Virtual participation can improve important aspects of the proposal vetting process; giving choice to prospective panelists can only increase acceptance rates among them, increasing the number of first choices among the experts, and reducing workload by those having to research and solicit prospective panelists. Again, these time savings get channeled into other important activities.

(5) Virtual participation does not diminish the quality of information necessary to make funding recommendations. Some nominal skills as a moderator are required to insure that all panel voices are heard, important issues debated, etc, but one could accurately assert the same need for nominal skills for all-physical panel moderators too. My perception is that virtual participants are every bit as well prepared as physical panelists and as attentive, and not harried or worried about catching a flight. I sometimes hear push back that there is “something about physical presence”, and there *is*, and much of its good, but its irrelevant for purposes of making science funding recommendations. Indeed, facial expressions, winks, and hand gestures are relevant to surviving together in the jungle, but if they are really nontrivial factors in panel recommendations, then respectfully, you are begging for over fitting the data.

(6) Virtual participation decreases ecological footprints. In almost all white papers on the environmental impacts of using information technology, the most common *proposed* POSITIVE impact is to offset footprints in other sectors, notably travel. And so why isn’t this actually happening?! This is the low hanging fruit of the promise of information technology, and if technologists don’t start exercising its promise, there is really no point in expecting others to do so. That said, I have a friend since kindergarten who is an airline pilot, a sister-in-law who is a flight attendant, a friend whose spouse is a pilot — facing the financial consequences for many people of changing lifestyles has to be part of what we worry about — but this worry can’t stop us from acting to change unsustainable lifestyles either!

Again, there are things important about physical presence that can’t be beat — at a minimum, the intellectual and social reward to a panelist, as well as the moderator — I loved dinner out with those few panelists that came to NSF for panels that I ran, Networking is important too, where it is possible that hallway talk will lead to an exchange of ideas and perhaps even collaboration. But this doesn’t happen much, certainly not to a degree suggested by theory, with hallway talk getting displaced by checking email and cellphone chatter. But if this networking were really a desired capability, then lets do it deliberately, do it by design, and do it virtually, in small groups and at regular intervals — why leave it to chance, as some accident of flying people across the country.

But in any case, I would hope that allowing choice to attend physically versus participate virtually was part of meeting designs in the future. Heck, if the timing was right, I could set up other meetings, sightsee and otherwise make good use of the trip, I’d go physically — I don’t know of anyone who likes slapping a back as much as I do.