Cesar working document 6 us experiment 4

CESAR WORKING DOCUMENT SERIES Working document no.6

Urban Strategy to support group learning Randomized controlled trial No4

M. te Brömmelstroet 13 April 2014

This working document series is a joint initiative of the University of Amsterdam, Utrecht University, Wageningen University and

Research centre and TNO

The research that is presented in this series is financed by the NWO program on Sustainable Accessibility of the Randstad: http://dbr.verdus.nl/pagina.asp?id=750

CESAR Working Document Series no. 6 Urban Strategy to support group learning

Page 2

TABLE OF CONTENT

1. INTRODUCTION ............................................................................................................... 3

2. SETUP OF THE EXPERIMENT ............................................................................................ 4 2.1 Intervention: Urban Strategy PSS ......................................................................................... 4 2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap? ................... 4 2.3 Setup of the controlled randomized trial ............................................................................. 5 Student groups ........................................................................................................................... 5 Control versus treatment ........................................................................................................... 6 2.4 Data gathering and analysis .................................................................................................. 7

3. EVALUATION RESULTS ..................................................................................................... 9 3.1 Perceived quality of the planning process ............................................................................ 9 3.2 Perceived quality of the planning outcome .......................................................................... 9 3.3 Effectiveness ....................................................................................................................... 10 3.4 Usability characteristics of Urban Strategy ........................................................................ 11

4. CONCLUSIONS AND IMPLICATIONS ............................................................................... 12 4.1 Reflections .......................................................................................................................... 12 4.2 General conclusions ........................................................................................................... 12

REFERENCES .......................................................................................................................... 14

APPENDIX I: COMPETITION SHEET (DUTCH) ......................................................................... 15

APPENDIX II: EVALUATION FORM FOR QUALITY OF THE PROCES ........................................ 17

APPENDIX III: EVALUATION FORM FOR QUALITY OF THE OUTCOME………………………………… 21

APPENDIX IV: EVALUATION FORM FOR USABILITY OF URBAN STRATEGY……………………….. 22


Page 3

1. INTRODUCTION

In this working document we report on a fourth randomized controlled trial with Urban Strategy to test the added value of this Planning Support System (PSS). Urban Strategy is a software package developed by TNO that aims to improve the planning process and planning outcomes of strategic planning. It does so by offering a range of quick models that show the effects of planning interventions in an easy to understand visual environment. To gain more insight into these potential improvements, we have conducted an experiment with a group of master students in Urban Planning of the University of Amsterdam. We make use of the measuring framework as introduced and discussed in CESAR Working Document No. 1. First, we describe the setup of the experiment (section 2). Then, the findings of the experiment are presented (section 3). In the fourth section we will briefly discuss the implications of these findings and further research.


Page 4

2. SETUP OF THE EXPERIMENT

2.1 Intervention: Urban Strategy PSS

TNO started around 2005 with the development of a PSS – Urban Strategy (Borst et al. 2007; 2009a; 2009b) – specifically aiming to bridge the existing flexibility-‐ and communication bottlenecks. Urban Strategy aims to improve complex spatial planning processes on the urban-‐ and regional level. To do this, different computer models are linked to a central database and interface to provide insights in a wide area of urban indicators and maps. The effects of interventions in infrastructure, land use, build objects and their functions can be calculated and visualized. Because the PSS is able to calculate fast and present the results in an attractive 1D, 2D and 3D visualisation this can be used in interactive sessions with planning actors. Starting point for Urban Strategy is the use of existing state-‐of-‐the-‐art and legally accepted models. To link these existing models a number of new elements were developed: - a database with an uniform datamodel; - interfaces that show a 3D image of the modeled situation, indicators and that offer functionality

to add interventions; - a framework that structures the communication between the models and the interfaces.

2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap?

The goal of Urban Strategy is to enable planning actors in workshop sessions to communicate their ideas and strategies to the PSS and to learn from the effects that are shown. This interactivity calls for fast calculations of all the model and fast communication between all elements. For this, the models were enabled to respond on events (urban interventions from the participants in the workshops. A new software architecture was developed to have all these elements communicate (figure 2). Through this increases speed and the wide variety of models that are linked together, the PSS aims to be highly flexible in offering answers to a large number of questions that a group of urban planning actors can have.

Figure 2 Schematic overview of communication architecture of Urban Strategy.

The 3D interface generates, based on objects in the database, a 3D digital maquette of the urban environment. To this, different information layers can be addes, such as air quality contours, noise contours and groundwater levels. Also, the objects can be colored according to their characteristics (function, energy use, CO2 emissions, number of inhabitants, etc). The 2D interface can be used by the end user (or operator) to add changes to the database. Objects can be added or removed, their


Page 5

location can be changed and the characteristics of the object can be changed. The 1D interface shows indicators that are calculated by all the models that are included. Examples are the percentage of noise hindrance, group risk in an area or the contribution of types of objects to CO2 emission.

Figure 3 The three interfaces of Urban Strategy

2.3 Setup of the controlled randomized trial

We designed our study as a randomized controlled laboratory experiment. With this we aimed to optimize the internal and external validity of our findings. Testing the most general claim of the PSS literature (that it improves planning) calls for a strong focus on the ability to translate our findings to theory. Although we have had special attention to mirror characteristics of urban planning practice as good as possible (see below), this means a sacrifice of ecological validity.

Student groups

The experiment was set up as an obligatory part of a second year course of the Bachelor Urban Planning at the University of Amsterdam in November 2012. A total of 78 students participated. They were informed that they took part in an urban planning competition. These students were randomly divided into groups of six. Within each group, each student was, again randomly, assigned one of six planning roles and received information about the plan (see below) that was relevant for his/her specialism and a specific goal for the planning session. Each group consisted of: -‐ 1 Environmental specialist (air quality) -‐ 1 Environmental specialist (External safety) -‐ 1 Environmental specialist (sound) -‐ 1 Urban Designer -‐ 1 Transport specialist -‐ 1 Project economist

Each of the 13 groups got the same planning challenge for an infill area in the old harbors of Rotterdam (figure 4). They were presented with an existing design for the area (figure 5) and the corresponding problems (each role had their own knowledge of specific problematic effects of the plan) and were asked to develop a new plan that would solve these issues as good as possible in a session of 60 minutes.


Page 6

Fig. 4. Brownfield location in the old harbors of Rotterdam

Fig. 5. Original design for the area provided to each group

Control versus treatment

The resulting thirteen groups were, again randomly, divided into six control-‐ and seven treatment groups. The control groups received no support; they were assigned to a table with empty plans, instructed to start, and informed that the time was up. These six groups worked simultaneously in one room. See figure 6 for both set ups. Fig. 6. Setup of control and treatment: PSS with surface table, chauffeur and process moderator (left) and the business-‐

as-‐usual table (right)


Page 7

The treatment groups got the full support of the abovementioned Urban Strategy PSS. In this experiment we have especially focussed on improving the process support. This is one of the recommendations of our ealier study: having more attention for the learning process of the group. Accordingly, we have structured the interaction between the PSS and the participants. First, they were asked to develop ideas for interventions (15 minutes), then they interacted with a service table and were presented with the effects of the proposed interventions (15 minutes). The remaining time was used to iterate between redesigning and reanalyzing the interventions.

2.4 Data gathering and analysis To find out if there are any systematic differences in the performance of the control-‐ and treatment groups we have made use of several data gathering techniques. In general, we used the framework for quality of planning as presented on the 2013 CUPUM conference. See table 1 for the dimensions and subdimensions that were measured.

Table 1. Dimensions of the quality of planning

First, two external planning experts (PhD candidates in Urban Planning of the University of Amsterdam) rated the general quality of the resulting strategies. For each strategy, they were asked to respond to statements on the dimensions A to F of table 1 (on a 7 point Likert scale). They were not informed of the hypothesis, nor were they aware of which strategies came from control-‐ or treatment groups. As a second way to assess the more specific quality of the outcomes, the effects of all strategies were calculated with Urban Strategy. The instrument adds specific knowledge to the planning process, with the aim to improve the strategies especially on specific subjects: external safety, noise, air quality, mobility and project finances. Without knowing which strategies were from control-‐ or treatment groups two Urban Strategies experts assessed how well the strategies met criteria for these five subjects on a 10-‐point scale. The heuristics used were:


Page 8

-‐ External safety: Bonus for minimizing number of houses in 10-6 contour (northern pier and head of middle pier), offices used as shield, displacement of sources.

-‐ Noise: Bonus for minimizing number of houses in 48Db contour, offices used as shield, effective use of deepening road, tunnels, screens, and noise reducing asphalt, reducing maximum speed, reducing number of severely noise hampered

-‐ Air quality: Bonus for explicit ideas on air quality improvements (there are no legal problems in the area) -‐ Mobility: Bonus for introducing new or strengthening existing modes, improving flow of traffic by

roundabouts or road extension, bridging the piers. Malus for lowering maximum speed, limiting access to or closing down streets

-‐ Project finance: Bonus for staying within budget of 2 million Euros, malus for displacement of external safety sources

Thirdly, all participants filled in an evaluation form in which we solicited their personal perceptions of the process. They responded to statements relating to the dimensions J-‐W of table 1. And fourthly, we have used direct observation: in real time (by a fourth person), and by video-‐ and audiotaping. These observations were mainly used to understand the outcomes of the first three analysis instruments. For the analysis, the responses on the statements were averaged and then compared and tested for systematic differences. To indicate the strength of the differences in effects, we used the p-‐value of a ANOVA F-‐test to compare two independent means (<0,05 is considered statistical significant). The statements were grouped for the subdimensions and overall dimensions by averaging them. The two statements on conflict were first inverted to make them compatible with this process. For the outcome dimensions, the scores of the raters were also averaged and then processed in the same way. Finally, we asked all participants in the treatment group to rate Urban Strategy on a number of usability indicators, using a 7-‐point Likert scale. NOTE: There were thirteen groups of which seven received our treatment. During the first treatment session, there were severe problems with the process and with the support by Urban Strategy. A quick scan showed that this significantly influenced the results of this group. For the purpose of this research, the results of this group were excluded from further analysis.


Page 9

3. EVALUATION RESULTS

3.1 Perceived quality of the planning process

Overall, we see a (small) positive effect of Urban Strategy support for the perceived quality of the planning process. In figure 7 and 8, all dimensions are displayed. The dimensions with a star have a statistically significant difference. The only dimension that scored lower in the treatment was cohesion.

Fig. 7. Scores for control and treatment groups on all proces dimensions (7-‐point scale)

Fig. 8. Differences between control and treatment scores (7-‐point scale)

3.2 Perceived quality of the planning outcome

The quality of the planning outcome (at right in table 2) is affected negatively by the support of Urban Strategy. Only the negative effect on implicational explicitness is statistically significant.


Page 10

Fig. 9. Scores for control and treatment groups on all content dimensions (7-‐point scale)

Fig. 10. Differences between control and treatment scores (7-‐point scale)

3.3 Effectiveness

Urban Strategy has a small positive effect on the effectiveness of the strategies. Especially the noise and mobility problems were better solved by the treatment groups (figure 11).

Fig. 11. Effectiveness (10-‐point scale)


Page 11

3.4 Usability characteristics of Urban Strategy

The participants responded to 16 statements (1 = very much disagree; 7 = ery much agree). From that, we see that the perceived Urban Strategy as fairly usable. Especially efficiency, clear output and supporting sketching and evaluation score well. The focus of Urban Strategy and the understanding of indicators score low. Figure 12 shows the scores per role. We see that the urban designer and safety expert are least satisfied regarding usability.

Fig. 11. Average usability scores of 42 participants (7-‐point scale)

Fig. 12. Average usability scores per role (7-‐point scale)

4. CONCLUSIONS AND IMPLICATIONS

4.1 Reflections

There are several important reflections to make on the methodology that have serious implications for the results and their generalizability. The laboratory setup with students allows for strong control over the treatment and control groups, but it also simplifies the complexity of daily planning practice on some important aspects (which is the target application of Urban Strategy). The most important ones are listed below. Students: In the context of planning practice, the student population of this experiment can be considered as being empty vessels. They do not have much tacit knowledge about the issues in the Waalhaven case; neither do they have vested interests or accountability for it. This makes the experiment relatively insensitive for some important effects that can be expected in real planning practice, such as developing a shared language or gaining consensus. Related to that, we would also expect to see more effects on the quality of the planning outcome in such a practical environment. Group size: Due to the limited amount of students, each planning group consisted of three students. This makes the experiment relatively insensitive for effects on group dynamics. In planning practice we expect much larger groups that generate much more (negative and positive) group dynamics. Then, we would also expect to see a bigger effect of Urban Strategy as either supporting or hampering these group dynamics. First timers: A large majority of the students stated in the open question that they enjoyed the strategy making exercise, since they never did this before. This is mirrored in the relatively high scores on the process dimensions (such as satisfaction) by the students in the control group. Again, this makes the experiment less sensitive for process effects that would be expected in planning practice. Here, the participants would have much more (negative and positive) experiences with similar processes and are better able to assess the added value of Urban Strategy. Also, they would be able to score the instrument characteristics (table 3) more in relation to other instruments. Related to that, we cannot assume that the control group worked as ‘business as usual’. Small population: 17 students (one person did not show up) divided into groups of three is a too low amount to control for other possibly important effects on the quality of the process and outcome. Although the students were randomly divided into control and treatment, it can still be that one or two very experienced students can influence the results. Raters: Due to time constraints, we worked with external raters from TNO. For them it was difficult to assess some of the quality dimensions of the planning outcome. This affected both their judgment of the control and treatment planning outcomes, which makes its implications less relevant.

4.2 General conclusions

With these methodological limitations in mind, the experiment has offered us some interesting insights on the use and usability of instruments such as Urban Strategy. First of all, the students that were supported with Urban Strategy, strongly appreciated the instrument. It is perceived as easy to understand, as supporting creating and evaluating ideas and as being transparent and user-‐friendly (all score above 5 on a 1-‐7 scale). This high usability score is a crucial prerequisite for the expected use of the instrument.


Page 13

Compared to earlier experiments, we found that improving the process support resulted in a significant positive impact of Urban Strategy on the perceived process quality. Although expected, it is interesting to see that by these small improvements all dimensions were positively influenced, with the exception of cohesion. There has been found no significant impacts of Urban Strategy on the planning outcome dimensions. This means that the external raters from Amsterdam could not find any differences in the quality of the strategies developed by the control and treatment groups. Again, although this can be partly explained by the starting position of the students, this is something that developers have to take into account.


Page 14

REFERENCES

Al, J., & van Tilburg, W. (2005). Basisboek instrumenten regionale bereikbaarheid. Rotterdam: Rijkswaterstaat AVV.

Couclelis, H. (2005). “Where has the future gone?” Rethinking the role of integrated land-‐use models in spatial planning. Environment and Planning A, 37, 1353-‐1371.

Dean, D. L., Hender, J. M., Rodgers, T. L., & Santanen, E. L. (2006). Identifying Quality, Novel, and Creative Ideas:Constructs and Scales for Idea Evaluation. Journal of the Association for Information Systems, 7, 646-‐699.

Friedmann, J. (1987). Planning in the public domain: From knowledge to action. Princeton: Princeton University Press.

Klosterman, R. (1999). The What If? collaborative planning support system. Environment and Planning A, 26, 393-‐408.

Lee, D. B. (1973). Requiem for large-‐scale models. Journal of the American Planning Association, 39, pp. 163-‐178.

Lee, D. B. (1994). Retrospective on large-‐scale urban models. Journal of the American Planning Association, 60, 35-‐40.

Rittel, H., & Webber, M. (1984). Dilemmas in a general theory of planning. In N. Cross (Ed.), Developments in design methodology pp. 135-‐144). Chicester: John Wiley and Sons.

Rouwette, E. A. J. A., Vennix, J. A. M., & Van Mullekom, T. (2002). Group model building effectiveness: a review of assessment studies. System Dynamics Review, 18, 5-‐45.

Te Brömmelstroet, M. (2010). Equip the warrior instead of manning the equipment: Land use and transport planning support in the Netherlands. Journal of Transport and Land Use, 3, 25-‐41.

Te Brömmelstroet, M. (2011). What do we support and how (well) do we do it? A multidimensional framework to measure the effectiveness of Planning Support Systems. Environment and Planning B: Planning and Design, submitted.

Te Brömmelstroet, M., & Schrijnen, P. M. (2010). From Planning Support Systems to Mediated Planning Support: A structured dialogue to overcome the implementation gap. Environment and Planning B: Planning and Design, 37, 3-‐20.

Vonk, G. (2006). Improving planning support; The use of planning support systems for spatial planning. Utrecht: Nederlandse Geografische Studies.

Vonk, G., & Ligtenberg, A. (2009). Socio-‐technical PSS development to improve functionality and usability—Sketch planning using a Maptable. Landscape and Urban Planning, doi:10.1016/j.landurbplan.2009.10.001

Waddell, P. (2002). UrbanSim: Modeling Urban Development for Land Use, Transportation and Environmental Planning. Journal of the American Planning Association, 68, 297-‐314.

Waddell, P. (2011). Integrated Land Use and Transportation Planning and Modelling: Addressing Challenges in Research and Practice. Transport Reviews, 31, 209-‐229.


Page 15

APPENDIX I: COMPETITION SHEET (DUTCH)

Introductie De Rotterdamse haven groeit nog steeds, maar deze groei speelt zich steeds meer ten Westen van de stad af. Omdat de havenindustrie steeds grootschaliger wordt, verschuiven havenfuncties naar nieuwe gebieden zoals de Tweede Maasvlakte. De hierdoor vrijkomende gebieden zijn aantrekkelijke locaties voor stedelijke transformatie: ze liggen relatief dicht bij de binnenstad en bieden in combinatie met het water interessante mogelijkheden voor wonen, werken en recreatie. Naast deze mogelijkheden zijn er uiteraard ook grote uitdagingen, zoals de aanwezigheid van industrie en afwezigheid van stedelijke infrastructuur. Eén van de gebieden die momenteel voor zo’n opgave staat is Waalhavens (zie kaartjes hieronder).

Figuur 1: Projectgebied Waalhavens in haar stedelijke context


Page 16

Figuur 2: Stedenbouwkundig plan (Oranje = wonen , blauw = werken, rood = horeca).

De gemeente Rotterdam heeft in samenwerking met het Havenbedrijf Rotterdam ambitieuze plannen geformuleerd voor dit gebied dat voornamelijk bestaat uit twee oude havenarmen. Het gebied moet een toplocatie worden voor de kenniseconomie, waar toonaangevende bedrijven zich willen vestigen en een leefomgeving bieden waar hoogopgeleide tweeverdieners willen wonen. Daarnaast moet het de meest duurzame stedelijke ontwikkelingslocatie van Nederland worden en voldoen aan eisen aan de milieukwaliteit. Zoals dat gaat, zorgen de beperkingen die het gebied kent ervoor dat deze bestuurlijke ambities niet eenvoudig te realiseren zijn. Het is aan jullie team om hier een zo goed mogelijke oplossing voor te ontwerpen die tegemoet komt aan het spanningsveld tussen ambities en realiteit! Het uitgangspunt daarbij is een voorlopig ontwerp dat gemaakt is door een onafhankelijk stedenbouwkundig bureau (figuur 2). In deze fase van het planproces gaat het vooral om de structuur, de plaatsing van de woon-‐ en werkfunctie in het gebied en de argumentatie daarbij. Je hebt in totaal precies 60 minuten om met je team tot een verbeterd plan te komen (dit lever je in met 1 kaart plus tekst). Zorg dus dat je dit goed organiseert! Je plan wordt uiteindelijk beoordeeld op een aantal factoren, zoals haalbaarheid (financieel en binnen milienormen), innovativiteit en concreetheid. De winnaar krijgt een TNO prijzenpakket.


Page 17

APPENDIX II: EVALUATION FORM FOR QUALITY OF THE PROCES

Ik#heb#een#goed#gevoel#over#de#sessie

De#sessie#heeft#geresulteerd#in#bruikbare#resultaten

Ik#was#goed#in#staat#mijn#mijn#rol#uit#te#voeren

De#onderwerpen#die#vanuit#mijn#rol#van#belang#zijn,#zijn#tijdens#de#sessie#goed#naar#voren#gekomen

Ik#heb#vanuit#mijn#rol#interventies#kunnen#inbrengen#in#het#plan

De#sessie#was#succesvol

Ik#ben#tevreden#met#de#sessie

Ik#heb#er#vertrouwen#in#dat#de#uitkomst#goed#is

De#andere#deelnemers#zijn#tevreden#met#de#sessie

Het#resultaat#biedt#een#echte#oplossing#voor#het#probleem

Het#resultaat#van#de#sessie#is#gebaseerd#op#correcte#aannames#over#het#stedelijk#systeem

Mijn#inzicht#in#het#probleem#is#vergroot

De#sessie#heeft#mijn#inzicht#in#de#relatie#tussen#de#verschillende#elementen#van#het#probleem#vergroot

Het#is#duidelijk#voor#mij#wat#de#oorzaken#van#het#probleem#zijn

Ik#heb#nu#meer#inzicht#in#de#processen#die#een#rol#spelen#in#het#probleem

Jullie#groepsnummer

Je#studentnummer:

Hoeveel#tijd#hebben#jullie#gebruikt:

Welke#rol#vervulde#je?MK:$ MK:externe$ MK: Stedenbouw0 Verkeers0 Plan0

lucht veiligheid Geluid kundige kundige econoom

NVT

1 2 3 4 5 6 7

Zeer$mee

eens

Jullie#groepsnummer

Je#studentnummer:

Hoeveel#tijd#hebben#jullie#gebruikt:

Welke#rol#vervulde#je?

Zeer$mee

oneens

De#sessie#heeft#geleid#tot#nieuwe#inzichten

Mijn#begrip#van#de#meningen#van#andere#deelnemers#over#het#probleem#is#toegenomen

Ik#begrijp#nu#hoe#andere#deelnemers#het#probleem#zien

De#andere#deelnemers#begrijpen#beter#hoe#ik#het#probleem#zie

Ik#begrijp#nu#de#voorgestelde#oplossingen#van#de#andere#deelnemers#beter

Ik#ondersteun#de#meeste#resultaten#van#de#sessie

Het#proces#heeft#me#inzicht#gegeven#in#de#meningen#en#ideeen#van#anderen#over#het#probleem

We#hebben#tijdens#de#sessie#een#gemeenschappelijke#professionele#taal#ontwikkeld

Z.O.Z. Z.O.Z. Z.O.Z.


Page 18

Tijdens(de(sessie(ontstond(een(platform(die(het(delen(van(ideeen(ondersteunde

We(hebben(een(gemeenschappelijke(visie(over(het(probleem(bereikt

De(resultaten(zijn(een(integratie(van(diverse(meningen(en(ideeen(van(de(deelnemers

We(waren(in(staat(consensus(over(het(probleem(te(ontwikkelen

We(hebben(een(gedeelde(visie(over(strategische(doelen(bereikt

We(hebben(een(gedeelde(visie(over(mogelijke(oplossingen(bereikt

Ik(had(een(sterk(groepsgevoel(tijdens(de(sessie

De(sessie(bracht(me(dichter(bij(de(andere(deelnemers

Er(was(sprake(van(conflict(tijdens(de(sessie

Er(was(conflict(over(de(uit(te(voeren(taak

We(hebben(de(tijd(efficient(benut

Heb(je(nog(andere(opmerkingen(over(de(sessie?

NVT

1 2 3 4 5 6 7

Zeer.meeoneens

Zeer.meeeens


Page 19

APPENDIX III: EVALUATION FORM FOR QUALITY OF THE OUTCOME

Geef a.u.b. het nummer van de strategie aan:

NVT

1 2 3 4 5 6 7

De strategie is ingenieus

De strategie is fantasierijk

De strategie is verrassend

De strategie is vernieuwend

De strategie is radicaal

De strategie heeft een transformerende potentie

De strategie is eenvoudig te implementeren

De strategie is sociaal acceptabel

De strategie is juridisch acceptabel

De strategie is politiek acceptabel

De strategie heeft een duidelijk verband met het probleem

De strategie zal het probleem oplossen

Dit is een effectieve strategie

De strategie kan worden opgedeeld in verschillende componenten

De strategie gaat in op "wie"

De strategie gaat in op "wat"

De strategie gaat in op "waar"

De strategie gaat in op "wanneer"

De strategie gaat in op "waarom"

De strategie gaat in op "hoe"

Er is een duidelijke relatie tussen voorgestelde acties & verwachte uitkomsten

de strategie wordt duidelijk gecommuniceerd

De strategie is makkelijk te begrijpen

De strategie sluit aan op stedelijke dynamiek

Heeft u nog verdere opmerkingen over deze strategie?

Zeer meeoneens

Zeer meeeens


Page 20

APPENDIX IV: EVALUATION FORM FOR USABILITY OF URBAN STRATEGY

Je Studentennummer

Welke indicatoren hebben jullie vooral gebruikt?

N.A.

1 2 3 4 5 6 7

Urban strategy is transparant voor mij

De communicatieve waarde van de output is hoog

De output wordt duidelijk weergegeven

Urban Strategy is gebruiksvriendelijk

De Output is geloofwaardig

Urban Strategy is uitgebreid genoeg

De focus van Urban Strategy

Het detailniveau van de kaarten is voldoende

Urban Strategy is makkelijk te begrijpen

Het instrument faciliteerde het evalueren van alternatieven

Het instrument faciliteerde het creeren van ideeen

Het instrument ondersteunde het schetsen van ideeen

Ik begrijp wat er (niet) wordt meegenomen in de indicatorenDoor Urban Strategy waren we in staat om het werk te doen met minder inzetDoor Urban Strategy waren we in staat om het werk te doen in minder tijdDoor urban Strategy waren we in staat om meer te doen in minder tijd

Heeft U nog andere opmerkingen over urban Strategy

Sterk meeoneens

Sterk meeeens

Technology

Cesar working document 6 us experiment 4