PREPARING FOR SITE VISITS:

THE AGONY AND THE AGONYMEREDITH MURR, UC SANTA BARBARA; SHERYL SOUCY-LUBELL, UC DAVIS; BARRY ROWAN, UC SANTA BARBARA

RANDY PHELPS, NATIONAL SCIENCE FOUNDATION

WHAT WE WILL NOT COVER

Anything Pre-notification of site visit:

How to assemble a good team

How to define the mission of the Center

How to construct a good center proposal, etc.

… for excellent guidance on these topics, please see articles in the Research

Development and Grant Writing Newsletter, published by Lucy Deckard and Mike

Cronan of Academic Research Funding Strategies, LLC

USING THE NSF STC AS A CASE STUDY

Site visits come in all shapes and sizes.

Varying in level of guidance, structure, on-site versus reverse-site, etc.

Not practical to talk about all the variations

Key take home message – it takes planning. Lots and lots of planning. More than you

think. It is AGONY. Be prepared.

Many of the issues we encountered in preparing for the NSF STC site visit are

applicable to all site visits.

But, follow the guidance and instructions from the funding agency/program officers.

STC OVERVIEW

SCIENCE AND TECHNOLOGY CENTERS:

INTEGRATIVE PARTNERSHIPS

Large NSF Center program ($50M in FY2012)

Budget $5 million a year for 5 years (renewable up to 10 years)

Funds innovative, potentially transformative, complex research and education

projects that require large-scale, long-term awards.

Expectation for partnerships among academic institutions, national laboratories,

industrial organizations, and/or other public/private entities,

Significant education and diversity mandates

Knowledge transfer, including technology transfer, providing key information to public

policy makers, or dissemination of knowledge from one field of science to another.

HISTORY OF THE STC COMPETITION

Solicitation Released: January 18, 2011

Preliminary Proposal: Due May 30, 2011

269 Submitted, 267 Accepted for Review

Grouped into eleven themes/panels

Panel review occurred September 27-28, 2011

40 Invited back for Full Proposals – Notification October 11, 2011

Full Proposals: Due February 3, 2012

11 Invited for Site Visits

Site Visits: Invited Late June 2012

September – December 2012

Blue Ribbon Panel – January 2013

5 Projects recommended for funding

No awards announced yet

WHAT IS THE MISSION OF THE SITE VISIT

For you to convince the site review team that the project should be

funded

They need to understand the project

They need to be excited about the project – that it has “Impact” and is

“Transformative”

They need to believe the project can be accomplished

They need to believe this is the right team to tackle the project

They need to think there are adequate resources and knowledge base available

KEY INGREDIENTS

The science has to be transformative – already assessed at earlier stages, but still important

PI – seen as a strong and competent leader

The whole team is/appears to be on the same page

Integration of all of the presentations – the mission, education, knowledge transfer and scientific themes

To win, everything has to be perfect

Digest the hints from the full proposal reviews and respond

SITE REVIEW – ADDITIONAL REVIEW CRITERIA

Is the budget appropriate for the scale, scope and complexity of the proposed

Center's activities?

Does the proposed Center management demonstrate the vision, experience, and

capacity to manage a complex, multifaceted, and innovative enterprise that integrates

research, education, diversity and knowledge transfer?

Is the proposed management plan likely to be effective?

Is the role of the external advisory board clearly and appropriately defined?

Is there an adequate succession plan for the leadership of the Center?

Are intellectual property issues adequately addressed?

PREPARATION

STC COMPETITION 2012:

GUIDELINES FOR THE SITE VISIT (FROM NSF)

STC COMPETITION 2012:

GUIDELINES FOR THE SITE VISIT - HIGHLIGHTS

The duration of the presentations and the agenda are fixed and are the same for each

site visit.

In planning the agenda, please allocate 30% of the time of each presentation for answering

any questions that the Site Visitors may have.

Do not plan any lab/facilities visits or demos.

Please make sure that you have appropriate representatives of your university administration present.

you must provide a password protected URL where you will post your proposal, all reviews, your

response to the reviews, Site Visit Agenda, list of participants, and presentation slides.

You must designate an administrative liaison person representing your team and e-mail his/her name

and contact information (including e-mail, tel. and fax nos.)

SITE VISIT AGENDA

Day 0

1:00 p.m. – 6:00 p.m. Site Review Team arrives at Hotel

6:30 p.m. – 8:30 p.m. Site Review Team Meeting

Day 1

7:00 a.m. – 8:00 a.m. Travel to Site Review Location, Light Breakfast

8:00 a.m. – 12 noon Introductions

STC Rationale and Goals

Research

(10:00 a.m.-10:20 a.m.) NSF Executive Session/Break

Research

Facilities and Physical Infrastructure (as appropriate)

(12 noon – 12:30 p.m.) NSF Executive Session

12:30 p.m. – 1:30 p.m. Lunch – Discussion with students (present students of the PI, co-PIs and participants)

AGENDA, CONTINUED

Day 1 Continued

1:30 p.m. – 3:00 p.m. Integrating Research and Education

Developing Human Resources

(3:00 p.m. – 3:30 p.m.) NSF Executive Session/Break

3:30 p.m. – 4:15 p.m. Partnerships and Knowledge Transfer

4:15 p.m. – 5:15 p.m. Administration and Management Plans

5:15 p.m. – 5:30 p.m. Wrap-up

(5:15 p.m. – 6:15 p.m.) NSF Executive Session

6:15.p.m. – 6:35 pm Critical feedback to the Pis, list of questions that require clarification on Day 2 at 9am

7:30 p.m. – 7:45 p.m. Transportation of the Site Visit Review Team to Dinner

AGENDA, CONTINUED

Day 2

7:00 a.m. – 8:00 a.m. Travel to Site Visit Location, Light Breakfast

8:00a.m. – 9:00 a.m. Meeting with Administrators Only (no PIs)/Institutional Support

9:00 a.m. – 10:00 a.m. Summary/Proposing team’s response to Critical Feedback

10:00 a.m. – 4:00 p.m. Site Review Team prepares Site Visit Report

Working lunch provided

THE NITTY-GRITTY

TO-DO CHECKLIST, A SELECTION (SEE FULL VERSION ON-LINE)

Administrative

Contact Information for everyone involved

Date Reservations – for site visit and red team review

Practice Schedule – how many in person, how many remote

Travel and Practices

Room – where is the site visit going to be held? Practices?

Food (for site visit and practices)

Reserve hotel room blocks

Technology

Logos and Powerpoint templates

Designate a laptop and projector for presentations

Website

Have multiple people keeping track of these deadlines so they don’t fall through the cracks

COMPLEMENT OF TEAMS – GEOGRAPHY COMPLICATES PRACTICE

All STC proposals have

One Lead Institution

Multiple collaborating Institutions (some could be international)

Geographic and time zones – hard to arrange schedules

Multiple external partners – travel remote practices

PEOPLE – FOR UCSB PROPOSAL

Research/Proposal team – 21 (from 7 different institutions)

Research Development at UCSB – 3

Honorary Research Development – 1

Administration – who came to site visit

UCSB Chancellor, EVC, Vice Chancellor for Research, Dean of Science, Dean of Engineering, Dean of Graduate

Division

Collaborating Institution #1, Vice President for Research

Collaborating Institution #2, Dean of Engineering

Graduate students and postdocs – from all participating universities

MAP OF ROOMS

ESB 1001 ESB 2001 ESB 2003

Capacity 100 Capacity 45 Capacity 25

Arranged with round tables Sides of room lined with chairs

Breakfast, lunch, PI meeting room All Presentations Site Review Team Breakout Room

PRACTICE SCHEDULE -VISUAL

Person 1

Person 2

Person 3

Person 4

Person 5

Person 6

Person 7

Person 8

Person 9

Person 10

Person 11

Person 12

Person 13

RED TEAM REVIEW

3 weeks before the site visit

Red Team Panelists

Science experts (from Advisory Board)

Education experts (from Advisory Board)

NSF expert (former STC program officer)

Campus leadership

All proposal team members AND grads came to UCSB for the red team.

FOOD AND THE NSF CONSTRAINTS

Money constraints (upper limit, hard in Santa Barbara!!!)

Location constraints (requested walking distance – nothing closer than

20 minute walk)

Conflicting advice about logistics???

Reservations

Payment – snacks but not meals, can only pay credit card or cash

DESIGN ELEMENTS TO MAKE LOOK COHESIVE

Logos

PI pictures

Group pictures

Powerpoint templates

WHAT WENT INTO BINDERS

Agenda

Presentation

Proposal

Review

Response

Pouch with pens, highlighter, page tags, and memory stick with all media

files

WHAT WE SPENT – TOTAL $36,000

Office Supplies: $500

WebEx: $200

Mock Site Visit

Catering: $4,000

Room Rental: $1,500

Travel: $5,500

Printing Binders: $1,500

Site Visit:

Catering: $12,000

Travel: $10,000

Media: $700

Vans: $200

Does NOT include extra person power – at least one-

half a FTE for 3 months

UCSB absorbed most of the costs – but that

discussion should occur early in the process

Travel reimbursements – good to have a standard

packet and it is clear what is needed (grad students are

bad at saving receipts).

LESSONS FROM THE SITE VISIT

YOUR NEW BEST FRIEND – THE TIME TIMER

Site visitors are happiest when they are

engaged and can participate (ask

questions)

Need to build in time for this – and

stick to it!!!

Practice transitions – they take time

too

Need strategy for cutting off questions

if they go long

BEWARE THE NIGHT OF DAY 1

Answer questions – until 2am – plan on it!

We were given 12 questions

Prepared a 10-page response

Have someone for a snack/beer run

Present answers in morning session

Opportunity to revise answers in the afternoon

Deadline was 4pm to give final answers to site visit panel

everyone needs to stay until 5pm on the second day – no noon flights

UCSB SITE VISIT REPORT – SUMMARY EVALUATION

We received the site visit report at 4:30pm on Day 2

Major Strengths

The Center envisions “Grand Challenge” research of the highest technological and societal significance.

The large-scale funding provided by the Center structure is required to accomplish the Center vision, i.e. to accelerate the development of solutions to the pending end-of-electronics-scaling challenges.

The Research themes are integrated and address transformational research goals.

The proposal is well focused, and the scope and objectives are well formulated.

High societal impact if successful; high transformative potential for US positioning in global semiconductor industry.

High potential for Legacy through technical contributions/spinoff projects across multiple disciplines.

Balanced risk portfolio with fundamental and practical objectives

Exceedingly high level of technical expertise and unique infrastructure with institutional integration;

Builds on USCB excellence in electronic devices and materials.

Exceptionally strong support from Deans of principal institutes.

Strong education and outreach supported by novel assessment strategy.

Major Weaknesses

No major weaknesses identified.

OUR SECRET WEAPONS

Our PI

Strong, Brilliant, Organized

Barry

Dedicated person for all logistics and general good nature (solid 3 months, 1st month ~50%, 2nd and 3rd month ~90%)

Barry was exceptional – you may need 2 people!!!

This was a strategic opportunity for the Office of Research

From UC Davis

Week leading up – social event at PIs house – good thing!!!!

Professional pictures taken at practice – so all looked same

SURPRISES – PLANNING IS NEVER PERFECT

Printer in the executive session room

Power and wi-fi access important for all site visit reviewers

Bad thermostat

Light yellowing in projector

Shuttle did not operate early enough to get proposal team members to campus

Binders were HUGE!!! And heavily used by site reviewers

Site Visit team never went to the room we reserved for them – the project team had to leave the main room.

Luckily we had another area big enough to fit us all.

More drinks – the sodas went fast

Don’t choose Monday and Tuesday for site visit dates. Last minute things on weekend.

THINGS THAT WENT WELL THAT WERE OUT OF OUR CONTROL

We had a great Chair of the site visit team

Food was good

Weather was good

No one was sick

No natural disasters (week before there was a fire in SB, two weeks

after was Sandy)

LOOKING FORWARD

Research Development Team – now has the experience of what a good

site visit looks like and can advise other groups

Research Development Team – received a lot of good publicity and

recognition, resulting in more requests for our services

ERCs anyone?

TAKE HOME MESSAGES

Preparing for a site visit is a BIG project, requiring more time and effort than you

imagine

As soon as you find out, evaluate your administrative needs – hire a person if feasible

Find the money to fund travel, food, etc. Who pays?

Who is the “presenter” – always theme leader? Other considerations? Language

Knowledge transfer? Diversity?

PI has to be seen as leader

Practice and preparation are the keys to success

Best way for the team to be cohesive and on the same page is for them to be interacting frequently

In-person practices important so that ALL team members know ALL parts of the project –

including assessment, education, knowledge transfer, etc.

Checklist for Preparing for Site Visits

Category Task When to Start Internal Deadline NSF Deadline

ADMINISTRATIVE

Contact information for everyone involved immediately

Designate administrative liaison immediately asap 6 weeks

Create budget for site visit and prep immediately within 1-2 days

Budget approval and spending account 3 months before within a week

Decide which materials should go in binders 2 months before within 2 weeks

Order office supplies for binders 1.5 months before

1 month before

Coordinate with printer (deadline for binders) 1 month before 2 weeks before

Dinner reservations for review team 1 month before 3 weeks before

Prepare info sheets on review team 2 weeks before 1 week before

Name badges for mock site visit 1 week before mock site visit

Food cost for review team 3 weeks before 2 weeks before ~1 week

Name badges for site visit 1 week before site visit

Assemble slides and prepare for printing 2 weeks before 1 week before

Take individual and group photos of project team

3-4 weeks before

1 week before

Put together binders 1 week before 2 days

Travel reimbursements for mock site visit when received within 2 weeks

Travel reimbursements for site visit when received within 2 weeks

TRAVEL AND IN-PERSON PRACTICES

Schedule Site Visit based on project team, Administration, and NSF Availability

immediately

Schedule presentation practice (based on availability of participants)

immediately within a week

Schedule mock site visit immediately within a week

Reserve rooms for site visit immediately within a week

Reserve rooms for mock site visit immediately within 2 weeks

Reserve rooms for practices as soon as dates are known

at least 2 weeks prior

Designate a person to time the presentations at the site visit

2 months before 3 weeks before

Reserve hotel rooms for mock site visit 2 months before mock

2 months before mock

Reserve hotel rooms for site visit 3 months before 3 months before 6 weeks

Reserve meeting room at hotel 3 months before 3 months before 6 weeks

Reserve transportation for review team 2-3 months before

2 months before 6 weeks

Travel information for mock and actual site visit 2 months before 6 weeks before 6 weeks

Travel reimbursement procedures 1 month before mock

3 weeks before mock

Catering for Mock site visit (know your policy) 1 month before mock

3 weeks before mock

Catering for site visit (know your policy) 2 months before 1 month before

Reserve furniture (tables, chairs, etc.) 2 months before 1 month before

Update all files on website 3 weeks before keep up to date 2 weeks before

TECHNOLOGY

Logos and presentation templates 3 months before 3 weeks prior to mock

Dropbox (or similar) for all involved (offline copy)

3 months before asap

Access to a speakerphone, phone lines, slide remote, laser pointer, etc.

3 months before a week prior to first practice

Designate a laptop for presentations 3 months before a week prior to first practice

Dedicated webinar and teleconference service 3 months before a week prior to first practice

Finalize agenda for NSF 1 month before prior to NSF deadline

3 weeks

Website – password protected 2 months before 1 month before 2 weeks

Wireless passwords (if needed) 1 month before 2 weeks before

Media rental (projector, laptops, power, etc) 1 month before 2 weeks before

Practice Schedule for UC Davis

Notification: June 28

Preliminary meeting of leadership with each Science, Ed, KT team: Weeks of July 2-13

Preliminary meeting with campus executives: July 16

Site visit date determined: July 16

Second meeting of Science, Ed, KT teams: Weeks of July 18-25

Logo and slide master finalized: Aug 3

First draft of presentations due to leadership: Aug 7

Second meeting with campus executives to discuss institutional match: Aug 13

Written response to reviewers: August 20

Confirmation from Mock Site Visit reviewers: Aug 22

Second draft of presentations due to leadership: Aug 27

Practice runs within Science, Ed, KT teams: Weeks of Aug 27-Sep 6

Third meeting with campus executives to discuss institutional match: Sept 4

Mock visit #1: Sept 7

PI travel to east coast partner institutions to meet with leaders and administration: Sep 8-12

All binder materials finalized except for slide presentations: Sep 12

Mock visit #2: Sept 21

Binders finalized: Sep 26

Dress Rehearsal: Sept 29

Site Visit: Oct 1-2

Practice Schedule of UC Santa Barbara Entire Center: October 5 and 6 at UCSB (on-site, in person): red team reviewers attending October 24 at UCSB (on-site, in person) Rationale and Goals, Theme 1, Theme 2 and Theme 3, Facilities: Phone conferences, plan for all day Dates: August 31 and September 19 Education, Outreach and Diversity: Phone conferences, plan for all day Dates: September 4, September 20 Knowledge Transfer, Industry Programs, Management, Evaluation: Phone conferences, plan for all day Dates: Dates: September 7, September 21

Person 1

Person 2

Person 3

Person 4

Person 5

Person 6

Person 7

Person 8

Person 9

Person 10

Person 11

Person 12

Person 13