Introduction - data.qld.gov.au · Web viewCCO suitability?? Public?? Data sharing policy aligning with licenses which are actually on the data sets. ... Solutions. Finding and accessing

ODIQ Open Policy Engagement Final Report

ODIQ Queensland October 2016

1. IntroductionThe Queensland Government and ODI Queensland have engaged with the broader community to help inform development of a new Open Data Policy for Queensland.

Open Data is a growing global movement that assists organisations to share data freely and openly and strives for transparency and innovation.

The engagement program, ‘Open Data, Open Conversations – inspiring a more informed Queensland’, sought input from a variety of stakeholders across multiple industries.

Articulous Communications facilitated four Deep Dive sessions with representatives across a number of Queensland sectors: Business and Start-Ups, Public Service, Spatial, Industry Organisations, Not-for-profits and Research, Academics and Students.

This report provides a summary of the findings from the Deep Dive sessions, including detailed information from users of Open Data that will assist Government to develop an Open Data policy that best meets the needs of the community.

The Deep Dives were supported by other engagement methods including:1. A preliminary survey conducted by the QLD Government from 21 July to 28

September through its open data portal to understand what types of open data were being used.

2. Social mapping on Social Pinpoint of opportunities and challenges around key themes relating to accessing, using, updating, sharing and reporting on Open Data.

3. Online community forum for discussion around key themes for the Open Data policy including managing data, licensing and attribution, engaging with re-users, quality of data, finding and accessing data, suitability and data stories.

Key issues to emerge from Deep Dive sessions: Need for agreed standards and consistency across departments Provide visualisation of data Develop a centralised catalogue Establish consistent metadata standards Ensure greater usability – provide both raw and aggregate data Provide more clarity around common licensing Promote achievements, undertake more engagement Refine the operational aspects of policy

From these key issues, some similarities and differences were evident across different sectors.

1. Start-ups and business were more concerned with ease of access to data and greater visualisation, and the need to be more engaged.

2. Government was concerned about the quality of the data, sharing of information across agencies, greater consistency, improved standards, and creation of a single catalogue.

3. Spatial, industry and associations were concerned with licensing and being clear about what can be published, metadata and consistency with standards, use of visualisation, and celebrating successes.

4. Research, academia and students were concerned with user interface and ease of access, ensuring both fine grain and aggregate data are available, better search facilities and standardisation of terms and formats, and improved feedback and more engagement.

2. Process: Deep Dive SessionsFour Deep Dive sessions were held with different stakeholder groups to discuss challenges, opportunities, and needs for Open Data in Queensland.

Live polling was conducted at the beginning of each session to get the group thinking more deeply about some of the issues around accessing, using, and managing open data.

Other activities encouraged groups to draw on their experience and knowledge of real life scenarios and possible solutions that would assist to develop an Open Data Policy.

The Deep Dives created an environment of knowledge-sharing and reflected the open data values of ODIQ.

All raw data from the live polling, and attendees input, are available as attachments to this report.

In total there were 82 participants over the four Deep Dive sessions.

This included representatives from 42 non-government organisations

Total non-Queensland government organisations represented in the Deep Dives included:

Session Total organisations

Business and Start-ups 22

Spatial, industry and non profit 12

Research Academia Students 8

The number of non-Queensland Government organisations providing feedback across all engagement was 79 – this included 42 for the forums, 20 for the online forum, and 17 for the social mapping.

At least 3 organisations from each sector provided feedback during the engagement.

Wednesday morning session (businesses and start-ups) had 16 participants with average online polling engagement at 77%.

Wednesday afternoon session (government) had 32 participants with average online polling engagement at 78%.

Thursday morning session (spatial, industry and associations) had 22 participants with average online polling engagement at 83%

Thursday afternoon session (research, academia, students) had 18 participants with average online polling engagement at 81%

2.1 Deep Dive Sessions Agenda

Session Details

Welcome and introductions

Welcome to participants.

The Project Overview of the Open Data Open Conversations Project including: What the Project involves Why We’re Doing it What We Hope to Discover What We hope to Design How We’ll Use the Information

YOUR EXPERIENCES:Live Polling

Live Polling to gather information on the group. Questions …1. How have you ever used open data? Yes, No2. What did you use it for? Open-ended3. What type of information would you be most likely to access with open data?

Health, transport, education, housing, business, employment, economic development, spatial, environment, justice and legal, local government, community services, recreation, emergency services, other.

4. What areas do you believe are the most important when it comes to using open data? Business improvement, increased knowledge, better service delivery, greater transparency, keeping up with global movements, being an industry leader, innovation, data management and quality, ongoing access.

5. How do you deal with multiple data sources and should there be a single source of truth? Open-ended

6. Given that globally more data has been created in the past two years than in the rest of human history, how can we maintain the currency of data? Open-ended

7. How can we best enable government agencies to release and update relevant data regularly? Open-ended

8. How concerned are you that open data can be used for purposes that are unintended or in a way we don’t like? Scale

9. Why did you answer this way? Open-ended10. How can we address the concern of re-identifying anonymised data? Open-data11. Is there a way to track and monitor what business and the community needs in

terms of data? Open-ended12. If we believe in open data should we decide that some data should not be

released because there is no demand? Open-ended13. Who is the owner of the data? (The government, The public, Those who use the

data)14. What’s one thing you would change to make open data easier to find, access and

use? Open-ended15. What is the greatest challenge when it comes to developing an Open Data Policy

for Queensland? Pin-drop on image16. What are the most important outcomes you’d like for your sector?

YOUR EXPERIENCES: Share your stories

Shared experiences of Open Data (small groups) Discuss:

Range of experiences What worked well What didn’t work well

1-minute reviews. Choose a few tables to share their experiences with the larger group.

Morning Tea break Video snippets of people taken.“One word” for open data

YOUR NEEDS: Challenges and Opportunities

Mapping the challenges and opportunities along an Open Data Journey using the following themes.

Finding and accessing data Data quality Suitability Licensing and attribution Managing data Engaging with re-users Implementation

YOUR NEEDS:Common Challenges and Opportunities

Each group to present key findings.

YOUR SOLUTIONS: Map solutions

Map out solutions to those challenges and opportunities that could be incorporated into the Open Data Policy.

SCENARIO TESTING

Each group to come up with an existing or future project they’d like to access open data for, or one they’d discussed earlier. Then take another group’s solution and test their scenario against it. Scribe up new ideas on the solution template.

Group debrief.

YOUR OUTCOMES

Open discussion

What are the most important outcomes you’d like for your sector?

Live poll (open text) or group discussion.

Next Steps Describe how the input will be considered

3. Live PollingLive polling was conducted at the beginning of each session and showed varying results across the four groups. Findings for the close ended questions are listed below.Key: Stand-out differences, stand-out similarities

Question Business and Startups Wed AM

Government Wed PM

Spatial, industry and non profit Thur AM

Research, Academia, Students Thur PM

What type of information would you be most likely to access using open data?

Top 3:Business 24%Local Government 12%Health 10%Transport 10%

Top 3:Transport 20%Spatial 15%Emergency Services 11%

Top 3: Spatial 29%Environment 17%Transport 15%

Top 3:Education 17%Health 15%Economic Development 13%

What areas do you believe are the most important when it comes to using open data?

Top 3:Innovation 27%Business Improvement 17%Greater transparency 15%

Top 3:Greater Transparency 22%Better Service Delivery 16%Data management and quality 14%

Top 3:Data management and quality 19%Ongoing access 15%Greater transparency 14%

Top 3:Increased Knowledge 29%Better service delivery 18%Greater transparency 16%

How concerned are you that open data can be used for purposes that are unintended or in a way we don’t like?

1. Not at all concerned (36%)

2. I’m neutral (29%)

3. Very concerned (21%)

4. Concerned (14%)

5. Slightly Concerned (0%)



3. Slightly concerned (21%)


5. Concerned (0%)

1. Concerned (40%)

2. I’m neutral (25%)Not at all concerned (25%)

3. Slightly Concerned (10%)



2. Slightly concerned (31%)


4. Concerned (13%)

5. Very Concerned (0%)

If we believe in open data should we decide that some data should not be released because there is no demand?

1. All data should be released (57%)

2. It depends on the type of data and the level of demand (21%)

3. Not all data should be released (14%)

4. Not sure (7%)


2. It depends on the type of data and level of demand (36%)


4. Not sure (4%)


2. It depends on the type of data and level of demand (32%)


4. Not sure (0%)


2. It depends on the type of data and the level of demand (13%)


4. Not sure (6%)

Who is the owner of open data?

1. The public (71%)

2. The Government (21%)

3. Those who use the data (7%)

1. The public (75%)


3. Those who use data (0%)

1. The public (79%)



1. The public (73%)



4. Key Findings

4.1 Deep Dive AM Session One: Wednesday 05 October, 2016: Business and Start Ups16 Attendees

4.1.1 Key Themes from Across the Session:

Easier access and user interface/user experience that better matches to how we find information online

Visualisation of data Need to engage in an ongoing way with the community Open by default Open data as a means for innovation Need for standards and consistency of open data Need for visualisation of data

4.1.2 Stakeholder Reflection:

Given that globally more data has been created in the past two years than in the rest of human history, how can we maintain the currency of data?

“Open access...the community keeps it current if it's valuable, Metadata and usable governance controls, Automated collection and transformation”

“Faster move towards digital govt, need insights specialists inside govt like corporates, govt consume its own data!”

“The future is in realtime data, Simplify access, Reliable IT systems”

4.2 Deep Dive PM Session Two: Wednesday 05 October, 2016: Government32 Attendees


Quality of data and how to ensure it is accurate Resourcing of open data Interagency sharing of information and how this relates to procurement of information,

consistent formatting, custodians Revenue impact of releasing government information for free that used to be sold Need for standards and consistency of open data Establish performance indicators for open data publishing for DGs and Senior

Executives, to incorporate this into strategic planning


Is there a way to track and monitor what business and the community needs in term of data?

“Socially engaging digital infrastructure, Profiling to understand needs”

“One point in and out rather than having to deal with separate depts, Use less jargon”

“Single point of contact. Visualisation of requests, Need to move beyond the spreadsheet”

4.3 Deep Dive AM Session Three: Thursday 06 October, 2016: Spatial, Industry Associations and Not-for-Profits16 Attendees


Potential risks associated with unintended uses Licensing of open data, and the need to be clear what is open data and what is

generally published information Need for visualisation of data Mapping of existing standards Celebrate successes and case studies of usage Metadata (use of metadata, type of metadata included and education of users to

understand what metadata is and how it informs them of the constraints of the open data available)

Federation of data standards / catalogues Measure the value of open data (including the financial benefits to business, plus the

opportunity cost of not maintaining open data) Better articulate and communicate the “why” of open data


How can we address the concern of re-identifying anonymised data?

“De-aggregate to the point where this is not possible”

“There will always be someone who can put two and two together. Difficult to prevent”

” Can't avoid it... so own it, Machine learning can already identify and de-anonymise data”

4.4 Deep Dive PM Session Four: Thursday 06 October, 2016: Research, Academia and Students18 Attendees


Access of open data which is easy to use Granularity. Need to access fine grain data, as well as aggregate data Need for semantic technologies to assist searchability by incorporating semantic (as

well as instructional) attributes to data. This might include the use of synonyms on terms, relationships between columns.

Standardised format for open data Common licensing Use a participatory approach to Open Data. Users to contribute and provide feedback

on what Open Data is available and in what format Need for raw data to be available and not just aggregate data. And that government

should be responsible for publishing raw data while users are responsible for cleaning data

Consider open data from the perspective of producer to user. Promote the value and benefits of Open Data through user feedback and success

stories Important to define the scope of Open Data and what needs to be an exception Aggregation of data needs to recognise individual privacy (maximum value vs maximum

protection) User experience / interface


What areas do you believe are the most important when it comes to using Open Data?

“Increased knowledge, Better service delivery, Being an industry leader”

“Better service delivery, Greater Transparency, Increased knowledge”

“Innovation, Ongoing Access, Data management and quality”

A number of attendees from across the sessions all identified Standards, Quality, Consistency and Clarity as their desired outcomes of the Open Data Policy.

5. Raw results Wednesday AM Group – Businesses and Start-upsOpen Data Template Responses: Challenges and Opportunities Activity

***Note some of the table fields have been left blank to indicate no response from participants***

Group 1

WED AM Final Challenges Opportunities

Finding and accessing data

AvailabilityFormalSilos of open data

Aggregating data and federating searchStandardsIncrease value awareness

Data quality CompletenessCurrencyReliabilityIntegrity

Open data standardseducation

Suitability Collected for specific purpose-not alwaysSuitable for other purposesLack of disclaimersBalance of sensitivities e.g. Flooding data impact real estate

Metadata (minimum)Disclaimers for data-why was it collected, how, when, refresh rateExtensible frameworks

Licensing and attribution

Lack of awareness of open data licenses and attribution

Consistent use of open data licensingFree data

Managing data Formats not machine readableCurrencyQuality controls

Open data standardsMetadataEducation in data management

Engaging with re-users

Feedback loopsNo need to know ‘who’ is using data but how and whyChampion data use across all levels of govt

Profiling user groupsCommunity of practicesEnabling feedback loops and data requests

Implementation Lack of expertise and knowledgeLimited open source options

Digital infrastructure with intuitive workflow/curation

WED AM Solutions

Finding and accessing data Standardisation of open data formatsFederating portal search engines to facilitate discoveryOpen APIs to facilitate searching portals

Data quality Minimum requirements for quality – reliability, currency, timeliness, accuracy, completeness, disclaimerData quality levels need to be identified against an agreed

standardLow data quality must not delay publishing

Suitability Frequency of data updates must reflect/support the data being collected and resources used to be allocated accordingly – i.e. real time data cannot be constrained/published daily/weekly/monthly because of resource constraints

Licensing and attribution Consistent data attribution using format or metadataData shouldn’t be open if it isn’t licensed

Managing data


Implementation Identify, support and promote implementation expertise through communities of practice and authoritative sources

Group 2

WED AM final Challenges Opportunities


Don’t know what is thereLittle context on a data setNo business problem-taggingGoogle search to find related data sets

*con-tags and meta data

Data quality ConsistencySource of truth

providence

Suitability No metatags-discernibilityData structure-mashing

Choice in download-cube wizardDatabase query tool on portal


Knowing what it means EducationRevenue streamDigital object identifier

Managing data Different data sets in different formats-consistencyDifferent data aggregations “by year/by month”Down to human created data sets-various skill/knowledge


Feedback from data consumers

Implementation CK on limitations Wizard

***No response from Group 2 for solutions

WED AM Solutions


Data quality

Suitability


Managing data


Implementation

Group 3

WED AM Final Challenges Opportunities


Narrowing down data sourcesWhat data exists? (traffic, population, crime rates, development)Availability of local government bylaws, costs

Google, business portals/enterprise centres, chambers of commerce

Data quality Currency, consistency, reliability Establishing standards for accuracy, rating of quality (e.g. gold, silver bronze), ability to edit data sets

Suitability Publisher and custodian knowledge (depending upon user)

Reuse existing data sources and processes


Managing data Data Hard to integrate LGA’s /SA’s

Able to edit data (editable)


Dealing with the government – finding who has data, whether government is sensitive to

Feedback/contact pointUser survey

requirements of those who need to use data

Implementation Automate the dataDependency on it

Small business data service – portable to access, with all data small businesses may need – as portal/mobile app

WED AM Solutions


Federated searches ‘Data.gov.au’

Data quality

Suitability


Managing data 1.


“communication”-just talk. Only solution to overcome ‘use of data’.Establishing processes -“social aspect”. This is seen as critical. Online forum for each data set.

Implementation

5.1 Attendees Storytelling Activity: What worked and what didn’t

5.1.1 Storytelling:Group 1: Validate procurement spend Human Services SpendGroup 2: Medical Research Institute Medical issues for returned servicemenGroup 3: Companies above a certain size across Brisbane

5.1.2 What worked:Group 1: Most reliable source Open Data Portal-Department of Communities Easy to find/UseGroup 2: Process Problem identification goodGroup 3: Quick responses. No data available (i.e. Brisbane City Council) ABS has data- not good

5.1.3 What didn’t:Group 1: Standards-hard to compare apples to apples

Group 2: Poor quality Different formats Difficult to mash together Time ConsumingGroup 3: Couldn’t find Not available for free Lack of knowledge

5.1.4 Attendees Feedback: What is the most important element/What should go in the Policy?

Problems are the same therefore we can apply same solutions Look at what’s worked inside companies and apply them to OD – continual evolution The users of data want to directly communicate with custodians “Standardised formats” – How do we choose & improve standards? Standardisation great but hard to achieve State Government doesn’t need to solve but looks at what others are doing Red Bull-sub sets, create a community, vertical partners and tags for research Many businesses are unaware how much data is available Accessing data for businesses, etc. can be made much easier through government

opening up to open data What measures do you put in place to make data effective? How do you allow open

data to evolve? Beta B Beta C /Channels of communication.

6. Raw Results Wednesday PM Group - GovernmentOpen Data Template Responses: Challenges and Opportunities Activity

Group 1

WED PM Challenges Opportunities


Inconsistent formatLack of a catalogueFinding the right data sets ‘the attributes’Need exists for subject matter exerts to advise consumers of data

Publishing services

Data quality Lack of a catalogueFinding the right data sets ‘the attributes’Need exists for subject matter exerts to advise consumers of data

Different levels Custodian identified eg. State, localCloud toolsets

Suitability Non-existentDifferent data agreementsEffort to make consumable

Good metadataEnd user requirements


Intended use of dataPoorly understood

Defined roles and licensing

Managing data Manual interventionIe. HoursMassage into DB

VisualisationBetter data management


Accepting end-user limitations Multiple re-distributionModel-value add and connectivity of datasets

Implementation All of govt-no STD platform Universal platform (common)Web enabledMachine, DB services, cloud, OData

WED PM Solutions


Mandatory to publish to open data-policy formulation

Data quality Filtering and cleansing toolsRatings (user review site)

Suitability Cyclical Shift Data-Info-knowledgePolicy to recognise this


Awareness campaign

Managing data Vocabulary, synonyms, dictionary- similar to BIM, spatial data, infrastructure


Not shared services arrangementOpen knowledge GP to roadtest/open data hub community to provide support and expert knowledge

Implementation

Group 2



Lack of a (federated) catalogue (discovery)Metatags too broad

Data quality Annual report datasets-individual agenciesData capture governance

Explanatory notesImprove/refine data capture process

Suitability


Managing data No authority-top down-resistance OD policy-governance, legislation, open by default.


How do we identify re-usersHow do we connect with re-users

Survey on portalAnalytics, stats, user reports-hits, downloads etc.Close the loop-meaningful metrics-measure outcomes-business case.

Implementation Shared data- date, naming, title, standards.Government use open data-increase usage

Data-INB management strategy

WED PM Solutions


Implement standard classifications

Data quality Concise data descriptionsClearly defined standards in the open data policy

Suitability


Managing data Open data policy that opens by default


Feedback survey on OD portal

Implementation

Group 3



Poor metadata (not intuitive/plain English)

Design a good search engine

Data quality Data is not great qualityPoor metadata

To not worry so much about itStandardise data qualityShared data testing lab

Suitability Difficulty in obtaining feedback, user information

Treat data as an asset – not administrative burdenGet communities to drive demandTailor data sets to specific outcomes


Managing data Strategic level inclusion Top Down management

Centrally mandated information standards Expansion beyond ABS


Go beyond the ‘app’Explore shared data/use shared data as a testing ground

Implementation Policy statement 1-2 pages easy to digestBreak down once and for all/silos

WED PM Solutions


Create a good search engine

Data quality Standardised data qualityEstablish WOG data dictionary

Suitability Reopen education/promotion of open data Research (ask them what it is they want to do)


Keep it free at all times

Managing data Top down driven strategy


Crowd sourcing/have specific release time and possible data source lsiting

Implementation Live feed of collected data directly to the data repositoryPolicy needs to be concise and easily understoodClear direction with principles

Group 4



Who owns cross department data? What data does gov have?Procurement of data from outside of agencyRed tape to release data

Use previously collated open data registers.Reducing gov cost-joint fund data collection across jurisdictions. WOG clarity around who owns what.

Data quality How do you measure quality? Quality is different for each data setIncreased media scrutiny over lack of data quality

Increased public scrutiny, benefiting govtBetter public serviceImproving resourcing ability

Suitability How to take government internal government data sets and publish in a way which is suitable for public consumption

Intermediaries (organisations in business of wrangling data) to convert govt published data for better consumption


Lack of knowledge and understanding of licensing & attribution

Managing data


131GOV- it doesn’t work

Implementation Lack of understanding Increase knowledge

WED PM Solutions


Improve portal-not user friendly, very boring. IS26 (CUE) locks down what gov can do with website

Data quality At a policy level-guidelines around data governance expectations

Suitability

Licensing and attribution Training Gov staff in open licensing and creative comms

Managing data Classification and standards to be applied to all datasets to drive ‘better’ metadata


Meaningful granular mechanism for consumers to (connect directly)with custodians in a timely fashion. NOT 131 GOV

Implementation

Group 5



No consistent taxonomyMany portals

Federated portalsCommunity led taxonomy development/open source Centralised tips & toolkits, SDKs

Data quality Staff hesitation (owner of data) exposurePoor governance BR data captureAccess to data custodian

Community scoringCommunity feedback

Suitability Lack of metadata (explain suitability, context)

Metadata standards


Difficult attributing derived data Lack of confidence in CCNot understood

Education, awareness of license without attribution/regulations

Managing data Updates, automation, creation of API (register)} source, origin, renewalFormats, availability

Improve data quality in org.Embed OD in procurement


Lack of resources, change management to protect end users of data, schedulingTrack usage/communication

Cross-community/jurisdictional collaborationConnecting with communityCommercialisation for startups

Implementation Ownership/resourcing/availability/marketingVisibility, prioritization, communicationEngagement/owners & publishers

More engagement with business for data publishing/sharing Identify and promote outcomes

WED PM Solutions


Grow standardisation approach from small (e.g. SEQ) to large (e.g. state) bottom up Start with scan of existing standards and then settle on one (then top down declaration) bottom up & top down

Data quality Data set centric crowd support/forum and user scoring both public and gov participation

Suitability State data quality risk as part of metadataEnsure minimum metadata (OD certificate)


Baden 2.0

Managing data Ensure business process relies on the data


SLA’s and use service management source desk tools for OD

Implementation

6.1 Wednesday PM Attendees Storytelling Activity: What worked and what didn’t

6.1.1 Storytelling:Group 1: Publishing? Managing? Using? Flood data: releasing map-releasing to publicGroup 2: Capital works program/budget Status-heat maps Level of progressGroup 3: Crime data/usageGroup 4: Open data general (revenue) How will changes affect revenue? Ie. land tax clearance searches If we open up, we won’t sell any more Lost revenue Warranty on access to data Business needs to be confidentGroup 5: Accuracy of data Agencies need policy in place to release data confidently Issues: relevance, budget and qualityGroup 6: Wildlife database Massive increase in requests for data

6.1.2 What worked:Group 1: Nil ResponseGroup 2: Visualisation (ie heat maps)Group 3: Visualisation (heat maps) Use of controlsGroup 4: Purchase a warranty (Gov sells) on search access/access

Group 5: Education will assist Need strong policyGroup 6: Publishing-making it open

6.1.3 What didn’t:Group 1: Can’t release to public Methodology not release Inter-agencies checking other data Collating initially-not everyone using the same gauge Nested copyrightGroup 2: Couldn’t publish, slowed money spent in certain areas and not others. Bottom line figures Scared to publish Transparency-political interference How do we separate?Group 3: Nil ResponseGroup 4: Quality Data Capture Need for definitions for usage-people may not understand Make definitions at deliveryGroup 5: Out of date data Liability-who is responsible Don’t know who is using itGroup 6: Maintaining system takes resources Limited funding People making money off it-but how do you support it?

6.1.4 Attendees Feedback: What is the most important element/What should go in the Policy? Open data by default – everything assessed for publishing How can state gov policy benefit Central place which covers questions from users and prescribe standards Predefined list of data sets to be published by local and state government Tie together with QLD Government Information Security Classification Framework &/or

state government archives Responsibilities of the data publishers Supporting material (e.g. guidelines, fact sheets, FAQs) Both IT and Business level Open resources (publications), not just data In some cases, we may not need to require attribution, in the interest of efficiency

(under status quo, you require attribution) Consider the connotations surrounding the quality of data when attributing data to

certain sources Whole of government API Key Ability to consume data without interference Guiding principles around quality (machine readable, etc) Methodology: Who, How What When Why? Releasing users’ algorithms Right for community to form projects (community insurance)

7. Thursday AM Group – Spatial, Industry Organisations and NFP’sOpen Data Template Responses: Challenges and Opportunities Activity

Group 1

THURS AM Challenges Opportunities


Difficult to find info from gov departments and authorities

Single access portalSingle query interface

Data quality Metadata standards-must publish planimetric/vertical accuracyData SourceGeometric errorsPlanimetric accuracy unknown, data source unknown

Suitability Geometric errors that prevent network analysisAttribute value rules not restrictive-ie.number fields defined as character

Tighten data modelsSystem to assess geometric data qualitySuitability statement


People don’t understand licensing restrictions.Difficult to know which licenses apply to which data.

Educate public on specific license requirements.

Managing data Letting users know when data is updatedLarge volume of data being created-sensors, IOT, satellite imagery

Some ways to register for data updates.


Risk of annoying usersDifficult to provide info in engaging ways.

Register for data updatesDynamically scaling cloud storageStorage systems scale dynamically based on real-time user behaviour/data set demand

Implementation Different data types.Different/restrictive IT architectures

Portal to access each department’s data-back end infrastructure maintained by gov and front end integrates with it.

THURS AM Solutions


Single point of access to all datasets

Data quality Metadata must include statements on: accuracy, source, assumptions,Capture data

Suitability Could include suitability statement in metadata.Data quality info in metadata should be comprehensive enough for the user to determine suitability.


Single access portal needs to prominently display licensing and Attribution.Single access portal to allow filtering on licensing restrictions.

Managing data All data hosted on data custodian infrastructure


Analytics of user interactions with the system

Implementation Metadata needs to be populated on all contributing datasets.System integrates with all contributing agencies.

Group 2



Need to know who to contact in organisationLots of data sets: finding what you need in a listsearch

Portals like QSpatial provide a possible method. Integration between sites and access points.

Data quality Maintaining accurate meta data.

Feedback mechanisms for custodians, crowdsourcing.Data ratings.

Suitability Making it suitable for external consumption in addition to current internal processes. Communicating limmitations of use

Community engagement


Complex System or process for simplifying the attributionsCreative comms

Managing data Not knowing when new data is released.

Streamlining


No register of users of files/services

Use of comments/RSS to notify usersVoluntary subscriptions for providersSocial media channels

Implementation Difficult to use the file typesLack of transparency of file/service content to determine suitabilityBroken links/changing dataLimited to specialist software

UsabilityOpen communication/dialogue with communityCelebrating success

THURS AM Solutions


Ranked list of results

Data quality Metadata standardsRating schemeTimes usedReview content

Suitability User-focused data collection workshops

Licensing and attribution Automated aggregation of metadata

Managing data Publishers-integration and federation. Min metadata standards. Users: comms channels


Subscriptions to updates/broadcastsSocial media channels

Implementation Min standards of usability

Group 3



Metadata kept to a standard and used

Searchable via google

Data quality Currency of data-don’t expect too much from free data.

Spend money improving dataEducate on metadata to interpret quality.

Suitability Address via metadata Highlight suitability in metadata tags.


Should this be a concern?Note:Lack of awareness surrounding licensing purposes and usesDerived data setsWorking out most efficient way to transfer data between peopleLiability

Can it be captured in metadata?

Managing data Update QA, QC-who is the custodian?

Becoming custodian


Communicating to anonymous users

Further refine product for re-users.Provide discussion boards for datasets.Tick a box to get updates automatically.

Implementation

THURS AM Solutions


Searchable via google.Use google technology to order search results.

Data quality Metadata used

Suitability

Licensing and attribution Share it like a millennial.

Managing data Must be funded and importance placed on filling in metadata.


Implementation

Group 4



Right to publish as open dataDefining usage

Supporting metadata, data dictionary, governance documentation

Data quality Linear data flow from capture to publishing Perceived benefit to improve quality where it’s not beneficial to data owners

Standards appliesPolicy supporting publishing process and linking to standards

Suitability Self-assessmentFit for purpose

Work flows-publishing


Definition of “open data”Understanding legal requirements/responsibilities for use

Clearly communicate licensing obligations

Managing data Time/costArchitectureLineageRisk assessment-approvals

AutomationGovernment policy and support management activitiesPublish the approval process, involves classification, risk assessment and approvals


What is wantedWhat doesn’t work

Collaboration-forumsSocial mediaNotifications-releases, modifications

Implementation Is it in a suitable format?AccessibleFor application as well as analytics/data crunching

THURS AM Solutions


Central portal-visual for ease of use and standard keywords (ANZLIC)

Data quality Meet quality standard before publication

Suitability Engage with end users at all stages


System of symbology for metadata/dd elementsEncourage/educate about licensing

Managing data


Funding incentiveCrowd funding for improvements, to a central body to then provide Grants.

Implementation Legislation/policy drivers to publish and/or use open data-incentivesand enablement.

Group 5



Private data accessible through gov portals. Eg using drone data and aggregating to single usable format

Proper negotiations around federation of metadata (e.g. private data federating with public portal)Making license restricted data still discoverable (licensing can be negotiated and copyright eventually expires

Data quality $ to address issues outside data ISO19157 Geographic information – data quality

Suitability Users not understanding the metadata elements (e.g. scale

Metadata reader eg. Colour code for accuracy level or data capture method


Appropriate levels of licensing-ignoranceGovernment procurement practices/policies

Learn off spatial efforts-GILF

Managing data Data processing and storage capacity

GA data cube


Trusting users-education Users providing feedback-improves data quality.

Implementation Working around the political panic reaction when something unexpected happensData culture-real metadata

THURS AM Solutions

Finding and accessing data Formalise the federation methods to metadata structures

Data quality Establish ISO 19157 in Govt

Suitability ISO 19195 in Govt

Licensing and attribution Government procurement practices/policies-standard licensing and at a Minimum whole of Gov.Any changes is by description only.

Managing data Open data needs to be a part of any department office.


Informative feedback loop to improve data quality

Implementation Fund it


7.1.1 Storytelling:Group 1: Policy rushed

Had to backtrackGroup 2: Connectivity/interactions Interface between government and community

Lack of knowledge on open dataGroup 3: Usage from client endGroup 4: Data access-how to find what you needGroup 5: Heart Foundation: Report that had admissions data related to particular medical conditions. Data supplied by hospitals and Institute of Health and Welfare.

7.1.2 What worked:Group 1: Nil ResponseGroup 2: Nil ResponseGroup 3: QLD spatial and globe ahead of NSW.Group 4: Good pockets of data but hard to find/no frameworks. QLD Globe AURIN-hosts 1000’s of data sets for research purposes. Can find what you need.Group 5: All came together in the end.

7.1.3 What didn’t: Group 1: Metadata not appropriate/complete. Poor data quality/no filter. Put everything up. Group 2: Inconsistency (better definitions needed). Blockage: when people approach-nothing happens-cycle of what to do next. Inconsistency made interaction between globes (eg. QLD, NSW) difficult. Group 3: Struggle with use of good vs evil. Sensitive data concerns (i.e. terrorism, telecommunication networks). How do we safeguard? Group 4: No framework. Vast majority of systems-hard to use/find. Group 5: Unclear about licensing of data and whether it is open or not (ie. Lots of data but what’s open data?).

7.1.4 Attendees Feedback: What is the most important element/What should go in the Policy?

Metadata standards Culture of metadata Awareness Thorough understanding and agreement throughout government and public about why

gov are using open data Why = it is a valuable asset Informed decisions around anything Removing hindrances to use Success measures Acknowledgment that the nature/value of data changes over time Understanding/appreciation of value & cost of the data Opportunity cost Policy communicated really well (e.g. understood by average person on the street) What’s the value of google maps?

8. Thursday PM Group – Research, Academia and StudentsOpen Data Template Responses: Challenges and Opportunities Activity

Group 1

THURS PM Challenges Opportunities


No standard portal/format i.e. disparate portals

Accepting third-party data

Data quality Trust-Multipart Raw Data-user

Suitability Purpose of publication of open data needs to be clearly defined

Feedback, Open forum


Common attribution CC.

Managing data Manage-Damager?? Common


Commonality Participatory

Implementation access Build from existing systems (e.g. AAF) Non-profit, research facilitation.

THURS PM Solutions


No Response

Data quality

Suitability


Managing data


Implementation

Group 2



Time required to find dataAccess due to privacy concernsfear

Collaboration with producers to make data available.

Data quality Coarseness of data-aggregation rounding, formats, metadata, state of dynamic, spatial

Standards and semanticsQuality assuranceData measures

Suitability User engagement /feedback


People don’t do it Simple licensing

Managing data Keeping data up to date and findable

Improves API, search engines, data analytics to find usage patterns


Lack of industry pull User engagement/feedback

Implementation $ Existing infrastructure

THURS PM Solutions


Semantic technology, pattern based, data indexing, visualisation, no SQL, location.

Data quality

Suitability


Managing data Analysis of interactive logs, cloud data and infrastructure.


Implementation

Group 3



Knowing what’s out thereSearchabilityAvailability and access to data-in real time (not by request)Customised search (analytics)

App to make searching easierSearch interface and decision tool to “pull at relevant data”Customised data execution tool

Data quality Metadata-detailed descriptions-deficiencies noted.ConfidencePrecedence in the data (how it’s collected and described)Openness of data (temporal/spatial)

Standardised way for data to be described (percentage)

Suitability Who should decide what data is suitable for release?How is it going to be used?Standards of data-can it be regulated by format, fields etc.

Define the scope of open data through policyTodd set a standard and quality of data in various formats


CC-BY 40 as standard fits outputsCCO suitability?? Public??Data sharing policy aligning with licenses which are actually on the data sets

To set the standardAligning policy and licensing and education.

Managing data


Implementation

THURS PM Solutions


Standard format in which data needs to be input (metadata description,Abstract-providing record, data dictionary). Provide a sample extract.Standard interface which uses a combination of keywords, relevance,and currency (timely)

Data quality Standard format

Suitability The scope of open data to be defined in terms of what expectations are.Standardisation.


A license of at least CC-By 40 on the data

Managing data A federal site to direct users to data sets-with min technical standards toprovide access to the data.


Implementation

Group 4



Determine the level of openness of data varies from one dataset to another.Downloading many files

Batch download of data made easier and more acceptable

Data quality Excuse used by agencies to not release data

Release clean data and describe how it was cleaned

Suitability


Applying AUS-Goal type of licenses uniformly and consistently

Managing data


Implementation Data.gov.au-host local government data or get a metadata feed.

Note on sheet: publishing raw data.

THURS PM Solutions


Indexing other data sets to simulate data in a rich context

Data quality

Suitability


Consistent licensing across data sets. Increase literacy of data publishers on licensing.

Managing data


Link output from re-use back to the source data.Link output from data back to gov policy and actions.

Implementation Co-hosting and catalogue.Consider metadata framework.Get state archives involved.Leadership from the top.Change perception of data as an administrative function to a core functionof each department.


8.1.1 Storytelling:Group 1: ResearchGroup 2: Access to coarse grain dataGroup 3: Access difficult. Coarse grain data difficult to applyGroup 4: Environmental data in fine grain form

8.1.2 What worked:Group 1: If they had confidence in the dataGroup 2: Coarse grain dataGroup 3: Web scrapingGroup 4: Getting access (going through custodian). Accessing data on site, before downloading. One single view of a greater amount of info (QLD Open Data)

8.1.3 What didn’t:Group 1: Lack of accessibilityGroup 2: No fine grain data. No standardised approach to releasing data. Hard to combine. Limited data preparation and reusability. Limited awareness about restricted data sets.Group 3: Difficulty accessing. Purpose of publishing unclear. Anonymised, coarse grain, aggregation-make data difficult to use.

Group 4: In fine grain form but limited file size (too much manual handling required to manage data). Aggregation loses individual meaning of data. Finding and accessing. Having to download prior to picking out important bits of data.

8.1.4 Attendees Feedback: What is the most important element/What should go in the Policy? being clear about what the policy is standardised format being clear about data quality policy not existing in a vacuum (aligning with national and international standards, to

retain value) different phrasing of the policy to be delivered to different audiences Access to the data is more important than what department is providing the data ‘one-stop-shop’ concept balancing personal data protection with the need to get data out to the public

9. Social mapping – Social Pinpoint ResultsA full list of comments is included in Attachment C

Word Cloud

Appendices

Appendix ADeep Dive Sessions Raw Data Results-Live Polling from all four sessions. Plus PDFs of Live Polling results

Appendix BOnline Comments from Online Forum

Appendix COnline Comments from Social Mapping

Appendix D

Documents

Introduction - data.qld.gov.au · Web viewCCO suitability?? Public?? Data sharing policy aligning with licenses which are actually on the data sets. ... Solutions. Finding and accessing