Japan 2010, Out of Time

In January 2010 I visited Japan on one of my last official trips while at NSF. I posted a summary of that trip on Facebook, which popped up as a memory today. I've copied that memory here -- its at the bottom. What appears first is a non-fiction travel log from ChatGPT, in the style of Steinbeck, which I requested. Its fun, but mainly instructive to me on writing style.

-----

Doug: Read this note <at bottom of this post--DF> that I originally had on Facebook, and write a non-fiction travel log in the style of Steinbeck that recounts the events, my impressions, and the like. Try to shoot for a 8 minute read.

ChatGPT: Here is a nonfiction travel log, written as a continuous piece in a Steinbeck-leaning voice—observant, humane, slightly amused by human machinery and human frailty, attentive to labor, ritual, and small grace notes. I’ve kept it grounded in your facts and events, but shaped it into a narrative arc rather than a diary. It should land around an eight-minute read.

---

Japan, Out of Time

It is late on a Friday night, nearly a week after I returned, and my body still refuses to believe me. It accepts meals at the wrong hours, demands wakefulness in the small dark of morning, and releases sleep only when it sees fit. This is not jet lag of the dramatic sort—no vertigo, no despair—just a long unwinding, as though some careful watch spring were loosening tooth by tooth. I have been back almost a full week, but part of me is still riding a train through fields I do not know how to name.

I left for Dulles at 4:30 on a Saturday morning in January, the house still dark, the world reduced to a cone of headlights and responsibility. By the time I arrived in Osaka on Sunday afternoon—4:20 by the clock there—I had been awake for something like a full day before departure and eighteen hours in the air after. Thirty-six hours had passed, but time itself had folded oddly, like a map creased too many times. Fourteen of those hours were not hours at all but arithmetic.

Dr. Numao was waiting for me beyond customs, calm and precise, and together we crossed the skyway into the hotel where the conference would be held. He would become Numao-san, and perhaps later Masa-san, because names, like distance, shorten with acquaintance. I slept from early evening until three in the morning, the kind of sleep that drops you straight down a well. When I woke, my mind clicked on at once, alert and uneasy. I worried slides into new shapes and rehearsed anxieties that would not benefit from rehearsal.

By four o’clock, coffee was no longer a luxury but a moral necessity. The front desk apologized kindly: nothing would be open for hours. But next door, along the wide airport road, was an all-night store called Lawson’s. Japan, I would learn, is mapped not only by streets and trains but by convenience stores—Seven-Elevens, Family Mart, AM/PM, Lawson’s—each block carrying one like a shrine to preparedness.

Lawson’s was bright and immaculate, stocked with food both familiar and baffling. There were sandwiches, hot dogs, beer. There was no coffee machine. I circled the store once, then again, considering whether to ask the young clerk a question that might take ten minutes and yield confusion on both sides. Then I saw the cans.

They were hot.

In refrigerated cases—unrefrigerated cases—were rows of canned drinks labeled Latte, Coffee, Black. When I lifted one, it was warm in my hand, like something alive. I turned cans over, hunting numbers I could recognize. “kcal,” “gr.” Seven kilocalories would not kill a man; seven thousand might. I gambled. One can said simply Black and promised 0 kcal. It was coffee, honest coffee, hot in a can. I bought two, for about a dollar each, and drank one immediately, standing under fluorescent lights at four in the morning, grateful beyond proportion.

Those cans would become my companions for the week—morning and afternoon, from vending machines on sidewalks, in subway stations, even outside a Buddhist cemetery. Coffee everywhere, quietly waiting.

The talk that Monday was the opening talk. It was not a home run, but it stayed in the park. I had given better talks, but only after scripting and practicing until the words could carry themselves. This was a new talk, on environment and computing, assembled in motion. Still, it must have landed well enough, because that evening the organizers asked me to give a speech at the banquet, and to offer the opening toast.

I do not drink. I said this. They acknowledged it and changed nothing. One does not wish to embarrass one’s country, so I said yes.

The conference itself was dominated by green chemistry and materials science—worlds adjacent to mine but not my own. During a break I asked an American chemist from a great university whether he followed everything. He admitted he did not follow much of the organic chemistry. This was a gift. It reminded me that incomprehension is democratic.

I listened with particular interest through the lens of cradle-to-cradle design—the idea that reuse must be total and ongoing, not a consolation prize measured in percentages. It seemed to me that we still imagine products alone, when in truth they exist in families and networks. Reuse, too, must be social. Materials must have companions, partners in reclamation, and the complexity of that dance may be beyond unaided human bookkeeping. One could feel the shape of a problem that would need machines not merely to compute, but to attend.

At the banquet, bottles of Kirin were lined up like soldiers. I listened carefully to the early speeches, relieved by their brevity. When my turn came, I thanked people, spoke of the long horizon of scientific work and the shared obligation to the planet. While I spoke, a waiter filled glasses. When I finished, he handed me one. I lifted it, said the toast, and then stopped.

One hundred people watched me expectantly. I had forgotten the final move.

“Cheers!” my host called out, remembering what I had forgotten. Everyone smiled and drank. I returned the untouched glass to the tray and spent the rest of the evening happily unburdened.

On Tuesday night, my host drove me into Osaka proper. The car spoke to us in a woman’s voice, navigating traffic with serene authority, rerouting us through back streets to avoid congestion. It was one thing to read about intelligent vehicles and another to be addressed by one. Once, the system erred—its map disagreed with the world—but it recovered quickly, apologizing by action rather than words.

I ride a Honda too, but it speaks differently. On full throttle it shouts joy. It was built for that. Low tech has its virtues. Later, in Tokyo, I asked to see the motorcycle lots at the universities, and my hosts indulged me. The Japanese build extraordinary machines. So do we. Beauty, like engineering, is widely distributed.

The next morning in Osaka I spoke to my wife on Skype while she sat in the Arlington library, unable to speak aloud. I talked; she typed. The call stayed open as I prepared for the day and she followed politics. Later she marveled at the geometry of it—Japan, Oregon, Arlington braided together by thin light.

The university visits were rich and generous—robotics, vision, language, learning. At the labs, shoes came off, slippers on. This felt right. In one Tokyo lab, the graduate students stood as we entered, and one barked a command. A chair scraped. A disheveled student emerged from under a desk and gave a clear presentation. Some constants transcend culture.

The bullet train carried me to Tokyo through dusk and gardens—so many gardens. Small plots everywhere, tended, deliberate. Even where homes had none, gardens appeared nearby, communal or shared. Food close to living. The train staff bowed when they entered and exited the car. Bowing was everywhere—mutual, repeated, unembarrassed. I felt the urge to bow deeply too, but hesitated, afraid of imitation without understanding. Still, the body learns.

In Tokyo I walked early, down to a Buddhist cemetery beneath my hotel window. There were offerings at the graves—coffee, beer, dice, green tea. I felt, as I often do in cemeteries, a calm belonging to the human line. The dead were not gone so much as placed carefully.

On my last night I attended a New Year’s gathering at an Italian restaurant, honoring a newly inducted member of the Japan Academy. There were speeches and toasts, quietly translated. I was urged to return to Kyoto. I hope to.

Before flying home, I walked for hours around the Palace grounds. Breakfast was food on a stick and canned coffee. Clerks spoke cheerfully to me at length. I said thank you. The day was clear and unhurried.

I will miss representing NSF when I leave government service—not the travel, which is tiring, but the vantage. To speak with colleagues around the world who labor seriously at shared problems is to feel hope made practical. NSF carries respect lightly, and that respect opens doors. No one spoke of competitive advantage. They spoke of responsibility.

Time at NSF has worn me down and tuned me up. When I return fully to academia, I doubt I will ever again be agitated by small politics. The work is too large.

It is late now. The body is nearly convinced. Somewhere, a vending machine hums, holding coffee warm against the dark.

 My Osaka host, Professor Numao, and his postdoc, Dr. Legaspi

 

A slide from my talk

My Tokyo host, Shinohara-san, of NSF Japan

Buddhist cemetery, Tokyo, about 20 floors below my hotel window

My coffee can collection

On a palace walk, the last day in Japan

My 1988 Honda Hawk 650 -- it talks too -- I've got some Shinto in me because of it

-----

Original Post from Facebook:

It’s late Friday night and I’m up, still out of synch with my usual rhythms after a week-long trip to Japan that ended last Saturday, almost a full week ago. It hasn’t been a hard adjustment back, as some report, just a long one.

I’d left for Dulles airport on Saturday morning, Jan 16 at 4:30 AM, and arrived in Osaka on Sunday afternoon, Jan 17 at 4:20 PM – a lot of that “36 hours” was due to the 14 hour difference in time zones, but I was also 18+ hours in the air, with a connection through San Francisco. When I arrived, Dr. Numao, probably Numao-san or maybe even Masa-san from now on, was waiting for me outside customs, and we walked across the bridge way to the hotel that was hosting the conference that I was speaking at next day. I’d been up a good 24 hours before leaving DC, getting ready, which included keeping fires at bay. I never sleep on flights and this trip was no exception, and I slept like a rock from about 6:00PM Sunday night until about 3 AM Monday (1 PM Eastern time Sunday), then got up to fiddle with slides and to worry :-).

Getting coffee was a big priority, and I got dressed and asked at the front desk – nothing in the hotel or airport for a few hours to come, but they suggested an all-night Seven-11 like store next door along a major thoroughfare, which turned out to be called “Lawsons” of all things – turns out there are Seven-11s, AM/PMs, “Family Market”, as well as Lawsons every block or so in the cities, and their street signs are about the only English you’ll see on a stroll, but Osaka airport is 40 minutes outside Osaka, on a built island, and this little Lawson’s was it. The place was bright, well lit and clean, a few familiar foods like ham and cheese sandwiches and hot dogs, beer and soda, but no coffee maker I could see. I circled the place, thought about asking the polite kid behind the counter, which would have been an interesting and probably prolonged conversation, as you can’t assume many speak English as you can in Europe, but I circled once more and in one of the refrigerated (not!) units, spotted “Latte” on a can – this was sort of promising, but I can’t take (literally) the milkshake like drinks that pass for coffee in Starbucks and the like, but it was worth looking into, and when I reached for the can, felt the heat – the can was hot (?!). There were in fact a slew of hot cans of Latte and (then) “coffee” labeled drinks too, and I checked them out in turn, looking for any comprehensible info on the backs regarding sugar content particularly – it was almost all incomprehensible to me, but here and there I’d spot “gr” and “kcal”, as in “7 kcal” – now 7000 calories in an 8 ounce can or thereabouts would kill you, so I thought “kcal” must be equivalent to a US “calorie”, or nearly so. Still, I wasn’t likely to find good ol’ black coffee in a (hot) can, that would be just too fantastic, but there it was … “black” … And on the back, “0 kcal” – fantastic, indeed !!!!

The two canned hot coffees I bought that Monday morning, for 120 Yen each (or about a buck and twenty) would be the first of many I’d buy in week-long early morning and afternoon treks in Osaka and Tokyo – vending machines with hot cans were everywhere too – every block, even outside a Buddhist cemetery -- so I could get my afternoon pick-me-up in subway stations or on the street on the way to appointments.

My talk later that Monday – the opening talk – wasn’t a homerun, but it went fine – I’ve hit homeruns before, but only when I script and practice them, becoming very comfortable with the material, its rightness, and the comfort allows a tempered enthusiasm to become the mortar that makes the delivery smooth and interesting, with confidently-uttered key points. This was something a new talk on the environment and computing, and I hadn’t had time to script and practice, but I gather it was good, in part because I was asked by the organizers to give “a speech” at the banquet that night (a speech :-O) and to give the banquet’s opening toast (a toast :-O) . I was about to balk at each, and I did mutter that I didn’t drink to the organizers (“You do not drink?” and no retraction of the request), but I wasn’t going to embarrass my country, so I said “yes” to each.

I wouldn’t have time to worry (too much) about the speech and toast, because the talks were interesting and very different from my areas – the conference, both Monday and Tuesday, was dominated by (“Green”) chemistry and materials science. During one of the breaks, I asked one of the US scientists, from one of our greatest universities, whether he followed all the talks, because the topics were highly varied, even if they were on topics that would fit into a very few academic departments. He replied that while he followed most of most of the inorganic, materials talks, he didn’t follow most of the organic chemistry talks … now, this is comforting to my ego – I’m serious.

I’ve become intrigued by the concept of cradle to cradle design ever since getting “the book” (http://www.mcdonough.com/cradle_to_cradle.htm ) from Mary Lou. As important as recycling efforts are, and I buy recycled when possible, if you are concerned that our species (and other species) be around and healthy centuries and millennia from now, or simply that your kids and grandkids are comfortable (or in my case, my nephews and nieces, and a grand-neice and brand-new grand-nephew!), 35% reclaimed materials just ain’t going to cut it; in fact 95% reuse ain’t going to cut it, and this also implies that one-time reuse, … nor 10-time reuse, etc will be satisfactory. It’s got to be full, high-grade reuse of materials over and over and over -- no more of a making, buying, and tossing it out mentality. So, the materials and chemistry talks were intriguing from this perspective. In particular, when we think about materials and/or products, it seems to be usually the case that reuse is considered as property of the material or product “in isolation”, where presumably AI could be used in the design-for-reuse process to determine what materials and products could be safely reused in conjunction, with reuse processes involving other chemicals such as solvents. I can conceive of material and product ‘companion’ groups (with group members being taken perhaps from different products!) that can be processed for reclamation in conjunction with one another – the reuse of which might even contribute materials to entirely different products. Thus, rather than thinking of manufacture and reuse of single products in isolation, we conceive of a graph/feedbacks of far greater complexity, where reuse of some products feed into manufacture of others … AI or like technology would certainly be necessary to manage and design such processes.

That night, as we all assembled in the banquet room, with bottles of Kirin beer nicely arranged on each table, presumably for the toast, one of the organizers called on a representative of a conference sponsor to give a “speech” – I was glad that I’d have a chance to see what was expected in a speech before having to give one myself, even as I secretly hoped that they would forget about me. After two relatively short speeches (“OK, good – I can do this”), the organizer called on me – maybe even using Fisher-sensei, or some other Japanese honorific (http://en.wikipedia.org/wiki/Japanese_honorifics ), “from the National Science Foundation”. I mounted the stage, thanked the organizers, the attendees, acknowledged my personal honor at being asked, spoke on the nobility of our scientific pursuits on behalf of the planet, the long years ahead … it was alright :-). During my speech, Kirin was being poured and glasses distributed, and a waiter came up to me on stage when I was done, and extended the full glass of beer, which I took, and said the toast, which was somewhat repetitive (but very abbreviated relative to “the speech”). When I was done with the words, I wasn’t quite sure what to do – I don’t remember ever giving a toast before, so I lifted my glass as in “ok, go ahead and drink up”, but didn’t think to actually say that. So, for one awkward moment, 100+ people are staring at me expectantly, and I’m clueless. Thankfully my host, who remembered that I didn’t drink, said loudly “CHEERS!” and everyone smiled and drank. The waiter had been behind me, as it turned out, and he extended the tray, and I put the virgin glass of Kirin on it, and enjoyed the rest of my evening with these good people.

On Tuesday night after the conference ended, my host drove me into Osaka, where I’d spend the night, prior to meetings at Osaka University the next day. On the entire trip his Honda talked to us, a woman’s voice in Japanese, rerouting us around Osaka to avoid rush-hour traffic, with the help of a high-quality dynamic display of our car’s whereabouts on an Osaka map, and verbal reminders of when to turn – we made the entire trip from airport to the center of Osaka guided by the car. While it’s one thing to read about intelligent cars, it’s quite another to ride in one, which I’d never done. It was an impressive demonstration of intelligent technology, the intelligence informed by reports on traffic conditions broadcast from other vehicles “downstream” of us. At one point the car misdirected us – it seemed to be clearly due to a mismatch in the roadmap database the car had access to and the actual street configuration – I could see the mismatch between the displayed map and physical scene in front of us, but only after committing to the route clearly indicated by a red arrow superimposed on the map – nonetheless, the car was back on track quickly, taking us through an impressive array of back streets to avoid heavy traffic, and when we inevitably hit traffic uttering “Be careful! There is a lot of traffic!” or so it was translated for me.

My vehicle BTW is also a Honda, but it’s never talked to me in words, though on the open road with the throttle fully opened the 647 CC engine screams “this rocks!!” That engine was built for the full throttle. It talks to me in other ways too :-). And like the best motorcycles, men of all ages and little kids can’t take their eyes off it (I loved Ka:ren and Roger's son's joyful yell on seeing a motrorcycle -- “Macaque!!” -- it's amazing, but little kids love them). And, you’ll never see a motorcycle rider road raging. All said, low tech has its advantages too. Later in the trip, when we stopped at Tokyo Tech and the University of Tokyo, I detoured us to the motorcycle lots, which my host in Tokyo, Shinohara-san, graciously allowed, after checking her watch . The Japanese make some of the world’s greatest motorcycles, and yes, America does too – I secretly long for a Harley Davidson V Rod (look at it: http://www.totalmotorcycle.com/photos/2005models/2005models-Harley-Davidson-V-Rod-VRSCB.htm; OMG!), possibly the most beautiful mass-produced bike ever.

At the hotel in Osaka, I caught up with Pat the next morning on Skype (about 4AM my time, after venturing out for hot coffee). I’d set her up on Skype a few weeks before so we could talk, particularly on international trips, but hadn’t caught her the previous days. When I logged on I saw she was on Skype too, at the Arlington library, I made a video call, and though she couldn’t talk (being at the library), I talked, she texted, and we kept the call open while I prepared for my day and while she listened to the political latest. Pat marveled later at being on Skype with me in Japan while simulataneously emailing my mother the latest in Oregon. Yes, indeed.

The Osaka University visits were very interesting, as were my later visits in Tokyo – I detailed all my lab and government (notably to a close analog to NSF, the Japan Society for the Promotion of Science, or JSPS) visits in my report back to NSF, but I won’t reveal the top secrets here :-). My hosts were gracious, interesting and interested in what was going on in the US – there is outstanding work in robotics, computer vision and language, artificial intelligence and machine learning, and informatics and ontology going on in Japan, with applications to eldercare, other healthcare, and security, among other applications.

Two asides: At the universities, taking your shoes off and switching to slippers was the norm in the labs – you’d wear your shoes in the main halls, but remove them before stepping on the carpeted labs, and slippers were set aside for guests. I told them that my wife and I observed this custom in our home and I think it put them something at ease in making the request. On a completely different note, when we walked into one of the University of Tokyo labs, the graduate students rose, and one turned back and called out a short bark .., a chair moved jerkily from a desk in the corner, and a disheveled young man rambled out from under the desk, recovering his wits, to deliver a competent presentation of Professor Kuniyoshi’s lab research on learning in infants, and simulations of it. Despite the many cultural differences it’s comforting to know that some things are common, like graduate student lifestyles.

My host in Osaka, Professor Numao, got me to Osaka train station in plenty of time for the bullet train to Tokyo, a 2-3 hour ride through the Japanese countryside, including a wonderful view of Mount Fuji in the distance at dusk (from the train’s left side on the Osaka-to-Tokyo run), though not that I was able to well capture on film. The most striking thing to me about the neighborhoods we went through were the large number of (vegetable) gardens. Even if a home didn’t appear from my vantage point to have a home garden, it seemed to be in close proximity to what appeared to be a neighborhood garden, perhaps sectioned, perhaps communal, perhaps both – I don’t know, but the gardens were ubiquitous on the long trip from Osaka to Tokyo.

On the bullet train I was a bit surprised that the attendants, those checking tickets and selling refreshments, would bow upon entering the car and exiting it and say some greeting and farewell too. I guess that I shouldn’t have been surprised, because bowing is pervasive, and it’s the real deal. As the conference was closing up on Tuesday night, for example, the organizers bowed to the support staff, and vice versa, probably two or three times. I found myself wanting to bow more myself, not the curt nods that I’d use, but deeper bows, but I think that a feeling of not wanting to appear a pretender, or perhaps not really sure of whether it was universally appropriate, made me hesitant, but the desire to “try it on” was real, and it wouldn’t have been long before I would have been bowing with the best of them, I’m sure, pairing it easily with the appropriate Japanese honorifics.

At my Tokyo hotel, I sent my suit (by now badly rumpled, after a flight around the world, two hotel moves and many meetings) out for cleaning/pressing before dinner with the NSF Tokyo staff, who I was glad were there. Like Osaka, Tokyo was filled with gracious people and interesting experiences. I was able to walk in the early morning, and checked out a Buddhist cemetery directly below my 21st floor room at the ANA Intercontinental. Like other cemeteries I feel belongingness to the human race when I’m in one, which may strike some as strange, but its peaceful nonetheless). This Buddhist cemetery was well filled, and it was interesting to see the things, presumably the items of endearment, that people left in remembrance of those passed on, or perhaps remembrance doesn’t carry the right connotation for what is intended in the placement of these items – a can of beer, of coffee, a bottle of green tea, dice, flowers of course, etc. And there are statues of Buddha (I presume) everywhere.

On my last evening, after my visit and presentation to JSPS, they asked me to their new year’s celebration at an Italian restaurant close by, which would also serve to honor a JSPS staff who had been inducted into the Japan Academy (“of Science”), or so I remember. There were speeches and toasts, which Shinohara-san quietly translated. My table mates suggested I return to see Kyoto, and I hope too.

The next morning, before catching the afternoon non-stop to Dulles, I walked about 2-3 hours to and around the Palace grounds, which Shinohara-san had point out on our ride the National Institute of Informatics two days before. I started the morning, like all in Tokyo, with food on a stick, which I relished each morning with canned coffee. At each of the convenience stores where I’d get food on a stick and canned coffee the counter clerk would speak to me at some length in Japanese, and though I had no idea what they were saying, they said it with a chipper voice, and I’d always say Thank You. My last day was beautiful, and though I couldn’t see the royal Palace because the grounds are just too expansive and walled off with moat, the long walk restful.

There is much else I could say about Japan – while I won’t detail friend Eric’s particular love of Japan, for example, it’s quite on target. :-)

One thing that I’ll miss when I leave government service is representing NSF and by extension the US on my international trips – it’s really not the travel per se, as I’m not a fan of flying and I really don’t have time for sightseeing except as flight scheduling allows. And each time an international trip approaches, I curse having agreed to do it, because I’ll tell you, I’m working longer hours than I ever have in my life, to include pre-tenure and as a graduate student, and there is always something to do at the office. But these trips are part of the eyes-on-the-prize aspect of my job, and once I’m there and talking to my colleagues in Japan, or with the OECD or European Commission, I’m struck by the dedicated people working worldwide to address humanity’s problems and it opens my eyes to hope and possibilities, and I have a chance to think deeply. There is also no doubt that I get a kick out of being with NSF – it’s respected worldwide, and I’ll miss the goodwill that comes my way simply by being associated with such an agency. I have never heard the phrase ‘competitive advantage’ at NSF – there is no talking out of both sides of the organization’s mouth so to speak. I have heard ‘competitive’ of course, as no one wants to be at the back of the pack, but there is a groking that the planet faces huge problems that can only be reasonably addressed if nations act in concert.

An advantage of going back to academia after being at NSF is that I can’t imagine ever getting worked up again about workload and politics :-), and the truth is my time at NSF has reenergized me, even as it wears me out regularly :-).

James K. Polk memory

James K. Polk, the 11th President of the United States, died 176 years ago on June 15. He was 53 years old when he died in Nashville from a persistent illness following the long trip home, just three months after leaving the White House. He accomplished all four goals that he set for his presidency, but much of this was in penultimate fulfillment of Manifest Destiny, and for this he is disparaged, even hated, by many. And I disdain that MD impulse too, even though California and Oregon are integral to my memories.

 

But what really intrigues me about Polk is not something I fully appreciated until the leadup to the 2024 Presidential election, for reasons you can probably guess.

Polk ran for president only once, won, promised not to run again, both before and after he was elected, and kept that promise. Remarkable. Amazing. Rutherford B. Hayes also fits the bill, but the election of 1876 is the most controversial in our history -- 2000 and 2020 don't come close, so some would add an asterisk to Hayes. I don’t, however, having read up on him – he was sincere, despite the complexities of his election, and of the times, Reconstruction. There are other near misses to Polk (and Hayes), as Patrick Winston, a founder of machine learning, would call them: notably Buchanan and Coolidge, and some might also add Truman and Johnson, but of these only Coolidge fits the spirit of why I hold high these three presidents – Polk, Hayes, Coolidge -- as undramatically stepping aside (as well as Washington, of course, and all who followed his lead before the 22^nd amendment).

So this Sunday I remember Polk, for his sins and for a quality I wish we better respected and held up as an exemplar to our leaders. Arthur Schlesinger, Jr, great presidential historian, wrote this, as quoted by John Seigenthaler:

"Neither Polk nor Truman was one of those creative presidents who make the nation look at things in a new way...But both had the intelligence and courage to accept the challenge of history. History might have broken them, as it broke Buchanan and Hoover. Instead it forced them, not into personal greatness, but into the performance of great things."

If you are looking for a connection to AI, which is all over my feed, talk to an LLM about all this. Despite their flaws, these AIs are typically the only entities that will spend an hour on presidential history with me, as well as formal language and automata theory, geography, notably water sheds, and Moby Dick.

Cancelled Flights, Biodiversity, Infinite Sums, Heart and Soul

This was originally posted June 12, 2009 at 7:48am, by the same name, on another blog platform.

I’m at Ithaca Airport – my flight delayed, but that means some time to think. There are times that an airline, and I’m thinking about one in particular, cancels flights and may chalk it up to mechanical problems when the truth is (I’m sure!) that the flight was under-booked. The “canceled” flight may be mine, which affects me directly, or it might be one “upstream”, which affects me indirectly as a flight delay. What’s odd is that when a flight is cancelled there is no statement about why it’s cancelled, and I don’t see anyone asking why it was canceled, at least not anymore. Sometime after taking the NSF job I actually went to the ticket agent and politely asked why the flight was cancelled and the agent was visibly surprised that someone asked!!?? After fumbling around, pecking at the computer terminal, the agent answered my query with “mechanical problems.” That’s amazing, because 20 years ago the agent would have announced up front why it was cancelled. Now, I don’t know for sure whether any particular flight is cancelled because of under-booking or perhaps the need for the airplane to serve a more heavily-booked flight which would otherwise have to be delayed, or the like, but thinking of it this way, be it true or not, has helped my serenity, because the canceled flight won’t produce Greenhouse Gas (GHG)! Carbon calculators will often reflect a savings for YOU if you fly less, but the only way that savings happen is if a flight is cancelled, and as people fly increasingly less, flights are eliminated entirely.

I’m now at Philadelphia Airport, my delayed flight having resulted in a missed connection, and now another wait. I’m serene. 

I had expressed concerns previously that biodiversity wasn’t on the map of the to-be-sustained from my experience at other venues, but MOST of the research talks at the Computational Sustainability Conference at Cornell reported on using computational and mathematical formalisms to reason about characterizing, monitoring, and protecting biodiversity !! Examples included the various approaches to addressing White Nose disease in black cave bats or fighting an atrocious disease mangling and killing Tasmanian Devils (which despite their name are awfully cute), estimating current and future distributions of a large variety of species, determining “optimal” policies for acquiring habitat, through purchases of private land, for the Grizzly bear, Canadian geese, Black Ducks, and other species (i.e., so-called ‘reserve design’), using computer vision to identify whale individuals, and GPS tracking to monitor seagull activity. A few hours ago, just before leaving, I listened to a coral biologist describe a differential equation model of disease transmission among Caribbean coral and their symbionts (new word for me) – the biologist is seeing rapid and pervasive evolution in coral (and pathogens!) through host-pathogen interactions, and this is but one example of moving targets in ecological research that makes computational and mathematical tools for analysis so important for handling complexity. Besides the ‘moving-target’ (aka dynamical) nature of ecologies, uncertainty in observations (how many geese were really parked at that lake?), and competing interests (the bear’s, the rancher’s, the tourist’s, the native American’s, the salmon’s, ….) as expressed by multi-objective functions, all make biodiversity questions complex. It’s a safe bet that the computer scientists and mathematicians are working on these questions because they ARE tough, BUT NOT TOO TOUGH, as they are problems that scientist’s can make a start at formally expressing. Nonetheless, the computational sustainability group isn’t satisfied with stopping at their current level of complexity, and a few economics talks are signaling anticipated scale-up to full-blown (human) societal questions.

I know that there are a fair number people that would not put biodiversity on their list of priorities, and frankly, if push came to shove, I don’t know where I would put it. There was one session yesterday dedicated to the social science of sustainability – the human factor -- with one sociologist saying that it was almost impossible to change someone’s ‘values’, but you had a better shot at changing their ‘beliefs’ (of facts?) – changing beliefs was a better avenue towards changing actions/behavior than changing values. You can perhaps show how values are connected – maybe, just maybe, there is a transitivity property of values.

Speaking personally, its good to know that there are researchers investing their intellectual and physical efforts in preserving other species – that some egghead scientist spends her or his life worrying about some slug that lives in only one lake on Earth, for example, may be a way that Providence implements a caring for that species – I’m glad for those people, even if I’m not one of them – I respect them – I want to help them. Perhaps that’s how a caring for a species piggybacks on my caring for a person??

It’s also been interesting to see how concern with sustainability is implemented in mathematical and computer models. Let’s say that we want a species to persist into the future – into the very long-term future. Ecologists told us that one way they represent this long term assumption was that they summed the *discounted* value of having that species around to infinity UNDER DIFFERENT CONDITIONS, and then they (e.g., USGS) would be biased to take actions that would lead to conditions that maximized the estimated sum of values (or rewards). What is meant by “discounted” is that today brings greater reward than tomorrow – for example, MY being above ground today is most important; my being above ground tomorrow is important too, but slightly less important that my being here today – if I’m not here today, then I can’t possibly be here tomorrow. Likewise, it’s important (to me!) that I be here day after tomorrow too, but its value is slightly less than tomorrow’s, which again is slightly less than today’s. Now replace “MY being above ground” with “Species X being EXTANT” and instead of a timescale of days, think of a time scale of years.

You might think that summing over an infinity of positive values, regardless of what they were, would equal infinity regardless of circumstance, thus there would be no way to distinguish the value of different scenarios (except those for which a value term for a year goes to ZERO aka EXTINCT). But so long as the discounting over the years causes future values to asymptote to zero (approaching zero, but never quite reaching it), the infinite sum will equal a finite value (and there are probably other conditions in addition to approaching zero that are required to achieve a definite finite value for an infinite sum). This is all intuitive to me, but still I had reservations with future reward approaching ZERO … could we have a mathematical conceptualization in which having a species around was NOT infinitesimally close to zero? And so on the last day I was affirmed (which is always good), when an economist got up and talked about non-discounted reward valuations that were possible (presumably, not based on infinite sums?)– the distant future did NOT approach zero value -- apparently well known stuff to some!! This was a great illustration of why this conference was such a good idea. 

Discussion following the economist’s talk got into issues of valuing the current generation (YOUR/OUR experience), the previous generation (PARENTS), to prior (Grandparents), the subsequent generation (Kids), and two generations forward (grandkids), and its interesting to consider whether mathematical models that value a window of five generations (your grandparents through your grandkids) would lead to policies with long-term benefit (because this window of five would slide down the generational stream so that while our generation doesn’t explicitly worry about our grandkids’ grandkids experience, eventually someone will). The whole discussion reminded me of the Great Law of the Iroquois that states, "In our every deliberation, we must consider the impact of our decisions on the next seven generations." (and on which the company “Seventh Generation” takes its name), and on a quote I heard at the OECD conference a few weeks back in Copenhagen: “we don’t inherit the planet from our parents, we borrow it from our kids”, imploring us to leave a better world to our children. In talking to one of the other participants, ML, we were also reminded of the social scientist’s talk of the day before about the near impossibility of changing values – but there might be critical points, corresponding to births (and deaths) of those we love where people are particularly open to value shifts – ideally, for the benefit of long-term sustainability of the planet we know and love.

Complexity, Computing, Citizenship

This was originally posted on June 7, 2009 at 9:33 pm by the same name on another blog platform.

Here is a quote from an article (http://www.news.cornell.edu/stories/June09/CompSustainConference.ws.html) on the conference I'm attending tomorrow:

"Conservationists will report on using computers to find the best balance of many competing factors in deciding, for example, which tracts of land to purchase to preserve a species, taking into account the budget, cost of parcels, economic impact on nearby communities and the best interests of the species. ... In projects like these, computers offer a way to try out thousands or even millions of possible scenarios, ..."

The scope of the conference is much broader than this one example, but all or most of the projects to be discussed will be looking at complex environmental and societal problems with multiple competing interests -- interdependent interests that have to be (or should be) balanced.

As my plane was touching down in Ithaca -- I mean just as the wheels hit the ground, I was reading this from "Soul of a Citizen" (Paul Loeb), quoting Wendell Berry:

"A bad solution is bad, because it acts destructively upon the larger patterns in which it is contained ... because it is formed in ignorance or disregard of them. A bad solution solves for a single purpose or goal, such as increased production. And it is typical of such solutions that they achieve stupendous increases in production at exorbitant biological and social costs."

and after more acknowledgement of Berry, and Loeb's inferences that "good solutions ... are part of a larger whole", Loeb goes on to say "We can contribute to the well-being of our society, the body politic, by applying a similarly holistic ethic of interdependence, and by listening to those whose voices are too often excluded from public discussion." (p. 131, Soul of a Citizen).

This reading was striking because acknowledging and balancing interdependent factors, concerns, and the like, is largely what this conference on Computational Sustainability is about, and using computer power to converge on and manage complex balancing acts which long-term sustainability will require.

I'm going into this conference, having just finished the conference of policy makers in Copenhagen on Information Technology and the Environment, with several "big" questions. One is what are the different meanings of "sustainability", if in fact anyone has a good definition? Exactly what are people expecting/wanting to be sustained? Of late, I've been disappointed that biodiversity doesn't seem to be something that is an explicit part of discussion -- it seems that as the environment worsens, values that we thought we wanted to be sustained fall away and we are "happy" with increasingly impoverished outcomes. This all fits the addiction model pretty well.

One point that I take from Loeb is that human decision makers ignore or deliberately remain ignorant of factors, sometimes, because they just don't know how to deal with them -- they'll even lie -- blatantly lie -- so that they can avoid the overwhelm caused by complexity. I'm hopeful that computing that enables us to deal with great complexity by processing millions of scenarios will also let us hear and consider all voices, be it the grizzly bears', the ranchers', the tourists', or in another setting, the homeless or the uninsured -- this is, perhaps, a humanistic possibility of computing.

Copenhagen OECD Conference 2009

This was originally posted on May 30, 2009 at 11:31 am on another blog platform as "Copenhagen 2009".

I just got back from Copenhagen, where I participated in the Organization for Economic Cooperation and Development (OECD) conference on Information and Communication Technology (ICT), Climate Change and the Environment. Last year I had been the only US government representative, and that had been a little scary, somewhat disappointing, and very cool – when they called on me for questions and comments they called on “United States” and before opening my mouth I reflected on what I was about to say :-). This year there were representatives from the State and Commerce Departments, and my (big) boss and I from NSF. I’m not revealing any secrets here – last year’s and this year’s are on the Web (http://www.oecd.org/sti/ieconomy/ictstheenvironmentandclimatechange.htm) -- more on this later and elsewhere.

I took the redeye from Dulles to Copenhagen late Monday afternoon, arriving Tuesday morning Copenhagen time. I was on Scandinavian Airlines (SAS) and I like it – every seat its own little screen with on demand video and music, a salmon dinner with not-quite-spaetzle pasta and lima beans (!?) to die for, power outlets for laptops, and stuff I’m forgetting. I watched two movies, "The Curious Case of Benjamin Button" (on the way out) and "The Reader" on the way back, both of which I'd balked at when they were in theaters, but I liked them each, particularly Benjamin Button, which was especially moving.

I took the train from Copenhagen airport to downtown, stowed my big backpack in a locker, only keeping my government laptop around my shoulder for safekeeping. Last year I happened upon the National Gallery of Art just before having to return, and wanted to see it especially this trip, but in no rush, and in fact exhausted but with really no option but to stay awake for many more hours, so I walked. 

<snip>

After a full morning, I took the train down to Helsingor, a beach town where the conference was held, did some brief walking around, talked to the tourist center, then a public bus to my hotel, a spartan place a couple of miles inland. I really wanted to sleep, but there was a conference reception that I thought I should attend for professional reasons, I wanted to stay up as long as I could for a better adjustment to the time zone, and frankly I felt isolated and knew connecting would help, and it did help.

The conference, in a nutshell, was focused on using information and communication technology (ICT) to reduce greenhouse gas (GHG) emissions due to humankind, and most notably what governments should do about promoting and supporting these efforts. There is great optimism that much can be done through virtualization (e.g., telework and conferencing with Web-based computer tools instead of the barbaric practice of flying cross country – yes, I recognize the irony :-), smart embedded systems (e.g., smart cars, smart buildings), which save energy through a large variety of computer-controlled mechanisms, and intelligent decision-making and planning systems that combine climate, economic and other social models. There is a lot of pushback though in implementing so much of this – for example, there are innumerable organizations that require that you keep your computer running 24/7 for reasons of “pushing” software and security upgrades, but my gosh, fixes to this should be 20 years old by now and the stupidity of designing such an energy-inefficient system is stunning -- it offends the engineer in me. I could go on and on with other forms of pushback and lack of awareness, and frankly, among other reactions, our failure to solve even simple, gross technological inefficiencies like the 24/7 computer-on policy has given me a certain sympathy for the disfunction that seems apparent in those trying to solve the planet's truly hard problems. I expect to be writing more on the topic of the conference. Generally, there is a lot that computer scientists and engineers can offer, both in terms of the products that they produce and in the ways that they think – my (big) boss Jeannette Wing has termed the latter “computational thinking”. 

It would have been fun having Pat with me to share the simple excitements and anticipations of a plane ride, train ride, castles, gardens, museums, and I hope that for next year as we are a good travel team, but next year I might be participating over the Web! So many decisions coming up, and even the “easy” ones can give me angst.

NSF's Broader Impacts Criteria

I attended a panel of colleagues, including a former member of the National Science Board (NSB) and three recent NSF review panelists, who discussed the revised broader impacts directives from NSF. It was an excellent panel, with great ideas, important insights, and pointers to helpful campus resources. In the latter half of this post, I offer some qualifications to some of the more nuanced comments, stemming from my experience at NSF (http://www.cccblog.org/2011/08/24/first-person-life-as-a-nsf-program-director/), but on the whole I was really struck by how much I resonated with what was said. I'll start by elaborating on these broad points of agreement, first talking about broader impacts as societal implications of the research, then broader impacts through formal and informal educational mechanisms.

 

Broader Impacts as Societal Implications

In my mind, the most important change to NSF's guidelines (http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_3.jsp#IIIA2 ) is that broader impacts (BIs) are to be evaluated by the same factors as intellectual merit (IM):

1. What is the potential for the proposed activity to:
    a. Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and
    b. Benefit society or advance desired societal outcomes (Broader Impacts)?
2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?
3. Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?
4. How well qualified is the individual, team, or organization to conduct the proposed activities?
5. Are there adequate resources available to the PI (either at the home organization or through collaborations) to carry out the proposed activities?

There are additional requirements for proposal content to ensure that sound assessments along all 5 factors of BI can be made, and of course, some of these 5 factors are newly applied explicitly to broader impacts.

The importance of "institutionalizing broader impacts" was made early by the panel moderator. A university that has advancing broader impacts in its bones encourages everyone to leverage and grow institutional resources, and creates a collective intelligence that isn't myopic about societal implications of science and engineering, even if individual scientists are often myopic.

When I was at NSF, most proposals didn't elaborate much on broader impacts. I think most PIs take on faith that their research will have broader societal significance, and don't feel the need or ability to elaborate beyond a phrase to a paragraph. Often I sympathize. For example, the PI working on a new computer programming language might feel that the broader impacts of that work are coextensive with all that is touched by computer programming! I am guessing that mathematicians and theoretical physicists are of the same mind -- that the broader impacts are so pervasive and sufficiently distant that its almost impossible to reason about and express. But particularly in the computing and engineering disciplines, someone should be thinking about the societal implications, because they won't all be positive.

Here are several more thoughts.

1) When a program director (PD) and/or a panel sees a proposal that elaborates intelligently on broader impacts, it really makes a proposal stand out from the rest. Occasionally, I've heard comments like "I have never weighted broader impacts so highly" from a panelist. A PD hears that and it makes a difference in the PD's recommendations for funding. For example, research in novel variations on mathematical and computational optimization that will be applied to ecological problems (e.g., design of wildlife reserves) or health problems (e.g., kidney exchange arrangements) are examples that would stand out, could be verified, and would be vehicles for describing the science and its motivation to the public, to include Congress -- a big plus, and one that I believe in.

2) One thing that I have never seen is an NSF proposal that considers the possibility of negative societal impact (together, we hope, with societal benefits too) -- for example, that increasing energy efficiency of a class of devices will cause those devices to be used more, and therefore the collective energy footprint of those devices worldwide will increase. If I ever did see such a proposal, coupled with some plan to guard against it or just test for it, I'd really be impressed, and I think it might impress (some) panelists too. As a PD, and more recently as a panelist on interdisciplinary proposals (e.g., Science, Engineering, and Education for Sustainability), I've seen good reasons to bring in social and behavioral scientists to an otherwise technical proposal because there are implications, often negative, to how humans interact with a technology.

3) In the case of multidisciplinary proposals, what is BI to one field might be IM to another, and vice versa. For example, a computer scientist working with an ecologist might propose to create a new sensor (IM to computing) that would enable better environmental data collection and analysis (BI for computing, and IM for ecology), and therefore better management of resources (e.g., water) for communities (BI for both computing and ecological science -- 1st-order BI for ecology, 2nd-order BI for computing). I find that this observation about the discipline-specific nature of IM and BI is generally new to PIs, and helpful in their starting to think about the IM and BI of a multidisciplinary proposal.
     Interdisciplinary teams generally can mitigate myopia (e.g., in the example above, consider how the "2nd-order" BI for computing can be traced through work with ecology, and these higher order BI effects can be negative as well as positive). Good for NSF for encouraging such proposals through funding programs! I think universities can do a better job of mitigating scientific and engineering myopia through interdisciplinary teaming, and this is NOT usually part of what many universities mean by "institutionalizing broader impacts".

4) Different divisions and programs of NSF view BI differently. The foundational areas (e.g., computer programming languages, computer hardware) are (almost by definition) farther from the broader societal impacts of the research -- after all, they are at the "foundation"! The PDs in the CISE Division of Computing and Communications Foundations will tell the PDs in the CISE Division of Information and Intelligent Systems "You ARE our broader impacts!!!" In the foundational divisions generally, dissemination mechanisms (e.g., workshops, published papers, etc) and education initiatives may dominate the discussion on broader impacts. This came out in the BI panel. I think that these differences will continue, at least I hope that they do, though I also hope we find mechanisms that allow scientists and public alike to appreciate the implications (i.e., 1st-order, 2nd order, higher order still) of foundational research to societal impact. This generally happens through anecdotal stories (e.g., the creation of the Internet, fertilizer that enables feeding the world, …), which is good, but many at NSF would like better longitudinal tools for visualizing impact of NSF's investments, through citation tracking and technology transfer, for example.

In my experience, measuring societal impact generally is not the focus of attempts to institutionalize broader impacts through "evaluation shops" and the like, except at the Center level -- but it can be.

Education, Outreach, and Diversity

 

Good mechanisms for broadening impact is through formal and informal education, where I would call much of what we call informal education to be "outreach". With respect to education components of BI, whether elaborated or not, most proposals I saw at NSF didn't aspire to broader impacts that went beyond the funding period. These proposals essentially proposed to do something worthy, but local, both regionally and temporally. Again, when you see ambition to institutionalize educational innovations so that they persist beyond the funding period and beyond the PI's immediate network, it really stands out. Here is where much of the emphasis on "institutionalizing broader impacts" (Google it!) can be found (Vanderbilt, OSU, Missouri, Stanford, etc). At Vanderbilt, the Center for Science Outreach (VCSO: http://www.scienceoutreach.org/) is giving PIs mechanisms for broadening the impact of their science through formal and informal education mechanisms. I expect that the Vanderbilt Institute for Digital Learning (http://www.vanderbilt.edu/vidl/) will work with VCSO and other groups for non-STEM, to further institutionalize broader impacts, insuring that positive BIs persist and grow.

The former NSB member highlighted the importance of evaluating BIs, just as PIs are expected to evaluate IM (see factor 3 above). This is fantastic -- I can't remember seeing a scientific evaluation plan for BI activities in proposals, except for large Centers where NSF required that an "independent" evaluation team for the BI aspects to be appointed. While NSF has been pushing on BIs for a long time, making BIs "first class" along with IM, is overdue.

I came back from NSF believing in the importance of institutionalizing broader impacts; there should be dedicated funds for BI (see http://www.vanderbilt.edu/provost/cms/files/Broader-Impacts-2-0.pdf) and particularly for medium and large proposals, there should be a co-PI who is explicitly named as the BI lead (my opinion); and some funds set aside to support communicating science and technology to the public too, because I haven't seen this latter activity explicitly called out. Apropos this last point, I spent late nights rewriting a fair number of award abstracts so that there was some chance that the research and the motivations for the research would be understood at some meaningful level by a larger public, including congressional staffers. While there were some notable exceptions, most PIs seem to think that they could let the proposal project summary serve as the award abstract -- sheesh! That summary might be a good starting point, but iteration is necessary to make it publicly accessible.

When I returned from NSF I learned about Vanderbilt's  Communication of Science & Technology major (http://www.vanderbilt.edu/cst/major.html); Vanderbilt must be (close to) unique in the nation in having such a major (good for Vanderbilt!), and it can be the basis for institutionalizing these kinds of broader impacts. Also, there can and should be a better connection made between the communications' teams at universities and schools with NSF, other agencies, and foundations. When I was at NSF, I can't remember ever getting award highlights from the professional science news writers who I know are writing for universities and schools -- why not?! Rather, again I had to iterate with PIs to get research award highlights that were informative and accessible to the public. In most cases, getting such highlights from PIs was like pulling teeth -- ugh!  Some probably don't value highlights much, while others would like to contribute, but they are busy too.  These highlights will be read by congressional staffers, and they need to be good, rather than some annoyance.

Related to the education components of BI, are diversity concerns, ranging from diversity of the research team, particularly on Center-level proposals, to diversity in future generations of scientists and engineers. Again, on Center level proposals there will be special accommodations to ensure that diversity and change in diversity over time is evaluated. But as with (other) education components, there was often little ambition and creativity in attention to diversity. Its not that broadening participation isn't an intellectually interesting area of study (e.g., see http://www.nsf.gov/pubs/2012/nsf12037/nsf12037.jsp), its that few PIs are thinking about it in those terms and so you read silly things, almost disrespectful in my mind, like listing the race and gender of selected members of the research team as the sole attention to broadening participation. In some cases you get the impression that the PI has put about 10 minutes of creative thought into broadening participation, and broader impacts more generally. Again, what are the ambitions for initiatives that move beyond the PI's institution and that will persist and grow after the funding period ends? Institutionalizing broadening participation concerns is germane here too.

Behind the Scenes

There was talk about "why don't PDs do this or that", "NSF should do this". There were some things that were said on the BI panel that aren't wrong per se, but some important factors don't seem to be appreciated. 

One of the most important things I learned at NSF was that there is substantive noise, and different sources of noise, in the process of vetting proposals; I don't mean that the noise is debilitating and that it compromises the validity of peer review as implemented at NSF, but its easy I think to "overfit" your experience on a panel and think you can prescribe simple fixes. Here are some observations.

1) BIs are historically weighted less than IM. In my experience, panels will judge a proposal worthy of funding or not based on IM, and break ties based on BI. The new guidelines won't guarantee equal weighting of IM and BI (see 4c of http://www.vanderbilt.edu/provost/cms/files/Broader-Impacts-2-0.pdf), and I don't think that they should, but I think that the new guidelines will insure that BI is more than a tiebreaker. In some cases, BI might be more heavily weighted than IM, and in a diversified NSF grant portfolio, I think that is perfectly fine. But again, recognize that IM and BI are discipline specific. As an aside, good grantsmanship would suggest that if you are getting declined for an education-heavy proposal in CISE (or MPS or ENG ...) then recast it and submit it to EHR!    

2) Review Panels are usually great at telling a PD which proposals are worthy of funding and which are not worthy of funding. This is already a big win for a PD who has to make decisions on what to recommend. In my experience problems arise when a PD PUSHES a review panel to do what it is not equipped to do. I do not think, for example, that a review panel is in a position to make hard recommendations (e.g., highly competitive versus competitive) based on projected funding levels. That's because the panel does NOT have all the facts in front of it to make such fine-grained recommendations.
         Funding levels are often much less than the percentage of proposals worthy of funding. This can lead a panel to "overfit" the proposals, with great angst over those last few proposals that are being placed in highly competitive versus competitive, and competitive versus not recommended. It's not that overfit will lead to "wrong" decisions or even "wronger" decisions (because most experts will focus on one valid set of characteristics over another valid set), but it can lead to great angst, and it can lead to odd factors for making the final hard calls (like who needs to get to the airport, an advocate or a detractor, of the proposal in question?).

3) One BI panelist said that on an interdisciplinary NSF panel that he/she had served on, 3/4 of the proposals were quickly decided because of IM weaknesses, and the remaining IM-strong proposals were placed in final categories based on BI factors. That sounds consistent with my experience, seems perfectly fine to me, but may seem less than ideal (aka overfitting) to some NSF panelists. Some additional points:

• (i) The new NSF guidelines may make proposal assessment more holistic (IM AND BI) throughout the paneling process, rather than IM assessment followed by BI assessment. Such a change may lengthen panel time.

• (ii) The weighting of BI is INCREASED in interdisciplinary settings. What would we otherwise expect an interdisciplinary panel to do??? Paneling interdisciplinary PRE-proposals relies even more heavily on BI factors. Its interesting to me that scientists agree with Congress on the importance of BI, when its not a proposal in their area.

• (iii) I once suggested to a PI who was not getting a proposal funded through the core program to recast it and submit to an inter-disciplinary, cross-directorate program, specifically to take advantage of the BI bias on interdisciplinary panels. Some might view this as exploiting noise (yes!) and some might say its one mechanism for getting out-of-the box research funded (yes!). The PI's proposal was recommended and funded under the interdisciplinary program; it had also been well regarded by previous core area panels, Competitive or Not recommended for Funding (yes, it can still be a good proposal in this latter case).

4) Not Recommended for Funding is not the same as not worthy of funding or not ready for funding. Again, we invite a panel to increasingly overfit the more we ask them to make finer-grained distinctions. Making finer-grained distinctions is more likely to tweak personal, professional, and scientific biases and constraints. I mean, why should charisma be a factor in making scientific recommendations? More importantly, why is NSF shooting itself in the foot by misrepresenting to the public and to Congress that some large percentage of proposals are NOT "recommended" because many will view this as NOT worthy, but this is NOT the case. At least some proposals that are not recommended for funding are, in the opinion of the panel, worthy of funding! Thereby we misrepresent the under-funding of science -- "but the expert panel said this stuff wasn't worth funding, so why increase funding!"
 
5) It's often the case that there is no consensus on the final, close call recommendations by a panel. This difference of opinion can and should be represented in a Panel Summary. If one or more panelists believe that a proposal should be rated more highly (and in any case), make sure that opinion and the reasons for it are expressed in the Panel Summary and that the PD has heard the argument during discussion (because of what I will say in the next bullet point about PD discretion). In fact a recommendation (HC, C, NRF) by the panel is NOT required (what's the PD going to do ? -- "make you" do something??!! -- no chance, only in your head). In one or two situations I had a panel split down the middle, and no one would budge on an HC vs C (for example), so they described the deadlock, and left the recommendation box unchecked. I had heard what I needed to hear to make a recommendation.

6) In my experience, NSF PDs are relatively quiet during review panels -- and I think that's a good thing. An NSF PD is not a DARPA PD, thank goodness, nor vice versa, thank goodness. NSF PDs have visions for their fields, but their actions are highly modulated by the research community, at least within their core discipline areas (PDs often branch out more when creating and implementing interdisciplinary initiatives that will influence their fields).
     A PD needs information for making recommendations, and while the panel recommendations are the single most important factor in a PD's recommendation, they are far from the only factor. Portfolio balance (where balance does not imply equal cardinality), institutional balance (ditto), PI balances (ditto), balances within the larger programmatic unit (e.g., robotics versus natural language processing versus …), …, AND WHAT THE PD HEARD DURING THE PANEL DISCUSSION. A good PD is a good listener. A good PD will likely speak up from time to time, but not too much. When I have seen what I regard as a PD stepping over the line and being too prescriptive, its been a rotator.
     In some sense it doesn't matter too much if a review panel "overfits" in its recommendations, because while a PD is very influenced by a panel, the PD is NOT tied to it. In fact, arguably the PD is there to compensate for panel overfitting, scientific conservatism, and bias. Its no small thing to decline a Highly Competitive proposal because you think a Competitive proposal should be funded instead (and there are not the funds to do both), and all this needs to be justified IN WRITING, so there is nothing flippant about all this. On rare occasions a Not-Recommended-for-Funding proposal may be funded (because that's not the same as Not Worthy) but that takes considerable justification.
    Thus, you might see a PD remain silent during the panel itself, because the panel is there for the PD to collect information, not about making final decisions. Should a reader advocate, in contrast, that a PD take a "leadership role" on the panel, for example on the importance of BI, recognize that that is a slippery slope. When I opened my mouth, it was most often to ask or answer a question, but yes, I would have to insure that the panel addressed BI to my satisfaction, that they wrote a respectful and informative panel summary, etc.
    That said, I think its a wonderful thing to set aside a session before the panel begins to talk to a panel about issues of intrinsic bias, broader impacts, etc, but once the panel starts, don't start (trying to) direct them TOO MUCH, else you won't know where they will go on their own, informed by the factors that they are in a position to assess, and thus a PD will confound her or his decision making process with the panel's decision making process.
    Would I advocate that we don't push panels to make the fine grained distinctions among those last close calls on the borders of categories (e.g., HC, C, NRF)? Sometimes perhaps, but suffice it to say that having a panel make the fine grained distinctions gets them to talk through the issues thoroughly, and its one mechanism for getting the issues on the table and heard by the PD, even if a PD might come down differently on the close calls than the panel does.
     But alas, there is another reason that PDs and their superordinates may push panels to make those final hard calls! Those final placements into HC, C, NRF are heavy lifting, and if the panel doesn't do it, the PD must. Its not that I think that the PD will do a better or worse job in those final placements (but might use different tie breakers than a panel) -- its that the PD often just doesn't have time. Exercising discretion, when you are (thankfully) obligated to justify it, takes a lot of time, which a PD often doesn't have.

Time, Time, and Time

Lots more I could say here, but let the following general points anticipate suggestions that "NSF" (as if NSF was monolithic) do this or that. Most NSF staff are working very long hours, and this includes a lot of in-the-trenches work. In the CISE (Computing) Directorate, I would sometimes think that if work weeks of more than 50 hours were made illegal, with stiff penalties for violators, there would be a year of extraordinary angst and pain for NSF and academia, followed by consistency, and organizational and programmatic sanity. Its only because of extraordinary hard work by many NSF staff that the whole system doesn't fall apart, but institutional performance is degrading, albeit gracefully. Any increases in funding to NSF generally go to new scientific funding programs, each of which increases overhead, and not towards increases in staffing. After getting back to Vanderbilt, I recall the excitement caused by the Robotics Initiative!!! And it was exciting. But you can bet that the overhead associated with it came out of the hides of NSF staff.

I've heard that NSF talks out of both sides of its mouth on broader impacts or on other issues, or that it drops the ball on this or that. Consistency requires training and that requires time. Going against a panel recommendation (supporting a Competitive proposal because of BI over a Highly competitive proposal) requires justification in writing, which requires time. Reading and pushing PIs for BI updates as well as IM updates requires time. Getting the "best" panelists to peer review proposals requires time, because in CISE at least, PDs will often get an acceptance rate from panel invitations of 20%-30%; I had high rates -- about 60-70% as I recall, because I allowed panelists to "phone in" (http://science-and-government.blogspot.com/2011/08/virtual-panelists-and-thoughts-on.html), but still, designing and recruiting and running a balanced panel takes time. And of course big thinking takes time, be it on designing funding programs along societal dimensions such as sustainability, health, and education; or tech/science dimensions such as robotics, computational game theory, etc.

A major constraint on NSF, or I should say staff within NSF, in responding to suggestions for "this or that" is time, time, and time. In addition to writing NSF, write Congress about funding of Science, and funding of the staff who create, implement, and run the  programs.

Rotating Program Directors at NSF

I recently commented on a blog post by Jeffrey Mervis on the AAAS Science blog at http://news.sciencemag.org/policy/2013/10/special-report-can-nsf-put-right-spin-rotators-part-1 , which acknowledged the pros of using faculty members from academic institutions as "temporary" or "rotating" program directors at the National Science Foundation (NSF), side by side with permanent Federal staff, but Mr. Mervis' article also points out that monetary savings might be achieved over the present implementation of NSF's rotator program.

I served at NSF as a rotating program director in the Computer & Information Science & Engineering (CISE) Directorate from 2007-2010 and have thoughts on the NSF rotator program. I repeat my comments to Mr. Mervis's article here, but I emphasized in these comments what savings might be most productive and doable; in addition, I think that some of the other recommendations of monetary savings in the Inspector General (IG) report cited in Mr. Mervis' original post seem less achievable or even less desirable -- maybe I will elaborate another day. I also argue that NSF should broaden its perspective on the possible benefits of rotating program directors.

-----

Your post (part I) and the IG’s report paint an accurate, though brief, picture of the IPA program: IPAs (and other staff) work hard and very competently, benefiting science and engineering research and education in the United States, but cost savings are possible. Of the suggested savings, reducing IPA travel back and forth between home institution and NSF would probably be (the most) productive. Frequent (e.g., weekly) travel by an IPA is costly, and it can also disrupt operations in NSF’s team-oriented environment. For IPAs who commit to a life predominantly in the DC area, I hope that NSF continues to pay for their relocation. However, for those who would prefer life predominantly at their home institution, let them telework, probably after an onsite orientation period that is designed to protect NSF esprit de corps. In either case, limit travel back and forth to some sensible number of trips, because 50 IRD trips a year is ridiculous, even if 50 days of IRD is not. This might also put NSF in a better position to negotiate for partial IPA compensation by the institutions of those rotators who stay at home (because the idea that NSF should expect home institutions to partially compensate IPAs who are working extraordinary hours for the government, and that's particularly true of anyone onsite at NSF, seems misplaced). Importantly, these arrangements are easier said than done, at least while preserving the benefits of the IPA program.

While I limited trips to my home institution of Vanderbilt University, I nonetheless ran two “virtual” review panels from my Vanderbilt office, supporting the IG’s contention (and many in NSF’s operational divisions too!) that much can be done through remote communication technology. And now we are getting into a largely underutilized advantage of the IPA program – that IPAs can benefit NSF operations as well as the scientific mission. IPAs are smart, usually very dedicated people who are watching and innovating the operations of NSF. For example, fully 3/4 of the review panelists that I recruited were virtual panelists – they participated by phone or video conferencing, and saved NSF substantial travel costs. My supervisors in the organization, including two IPAs, supported this activity. Other IPAs innovated in similar ways, as well as some members of the permanent staff. If NSF made a commitment to supporting IPAs who had a desire to telework from their home institutions, with protections in place to protect high-quality communication, responsiveness, and NSF esprit de corps, it would go a long way to building a culture in which much larger monetary savings could be realized through the use of virtual panelists (http://www.sciencemag.org/content/331/6013/27.full ), as well as reaping other substantial advantages of virtual panelists (http://science-and-government.blogspot.com/2011/08/virtual-panelists-and-thoughts-on.html )

Apropos the possibility of operational benefits of IPAs, exit interviews of IPAs seemed spotty and certainly not universal when I was there. It strikes me as a terrific lost opportunity if NSF is bringing in talented faculty members, almost all of who have the luxury of speaking their mind because of job security that stems from tenure, and not exit interviewing them and then acting on those interviews!

The IG report also suggests the desirability of a person or office dedicated to evaluating the IPA program on a continuing basis – that is a terrific idea. I have no doubt that ongoing evaluation would affirm the scientific advantages of the IPA program and improve IPA management. In particular, John Conway’s article alludes to the “cultural” differences that often exist between academia and the team-oriented environment of NSF. An IPA-oversight officer who respected and appreciated the IPA mission would presumably help define best practices of IPA orientation, training, and management to effect the transition to the NSF environment, as well as evaluate the program.

Finally, part 2 (http://news.sciencemag.org/people-events/2013/10/special-report-can-nsf-put-right-spin-rotators-part-2 ) of your article highlights a case where an IPA may have been powerless and dismissed summarily. I do not know this case, but five comments seem relevant and responsible: (1) I was proud of NSF’s policies and practices regarding conflicts of interest (COI), and I wish they were standards practiced throughout our Federal government; (2) my experience was that the professional ethics officials at NSF were honorable, highly competent, and responsive to requests for clarification and other help on COI issues; (3) the COI standards are high (thus my pride), but I would regard a case like that outlined as forgivable and correctable in a gentler and more constructive fashion than that described -- I can imagine circumstances in which I might have missed real or perceived COIs too; (4) if there were an officer responsible for assessing the IPA program at NSF, then presumably they would have looked carefully at the actions of all IPAs involved, including supervisors, and made corrective recommendations on IPA training and management at all levels; and (5) the individuals within NSF best placed to speak out on any injustice might well be IPAs, again because of the job security that stems from tenure at their home institutions. That’s not to say that rotators should be watchdogs, but more thought should go into how to use IPAs effectively to inform operations and management, as well as science.