Marketplaces for Data: An Initial Survey - Plone sitemontesi/CBD/Articoli/MarketPlaceForData.pdf · any form ( e.g ., ), or data enrichment tools ( e.g ., attensity.com). In order

Marketplaces for Data: An Initial Survey

Fabian Schomm1 Florian Stahl1 Gottfried Vossen1,2

1European Research Center for Information Systems (ERCIS)University of Muenster, Germany

[email protected]

2Waikato Management SchoolThe University of Waikato, New Zealand

ABSTRACTData is becoming more and more of a commodity, sothat it is not surprising that data has reached the status oftradable goods. An increasing number of data providersis recognizing this and is consequently setting up plat-forms for selling, buying, or trading data. We identifyseveral categories and dimensions of data marketplacesand data vendors and provide a snapshot of the situationas of Summer 2012.

1. INTRODUCTIONToday information is one of the crucial driving

factors for most businesses. Only if high qualityinformation is available, correct decisions (i. e., de-cisions in the interest of company revenues) canbe made on a rational and well-founded basis. De-spite the sheer quantities of data available on theWeb, such information is not always easy to find,and data marketplaces, surveyed in this paper, areone of several recent developments to remedy thissituation.

Shortly after the arrival of the Web in the early1990s a new category of professionals emerged whotook on the function of information intermediaries.To these intermediaries search task could be given,who would then search the Web correspondingly(for a fee) and return the results found. In 1998 theterm data marketplace was probably first used byArmstrong and Durfee [1], who modeled trad-ing of information between digital libraries, focusingon the motivation and behavior of participants andidentifying factors that affect cooperations in a net-work.

Thanks to advances in technology, but also tothe vast amount of data available nowadays, nu-merous new forms of marketplaces for data haveemerged. A modern information intermediary orinformation marketplace in our understanding is aplatform through which data can be purchased or

sold. Commonly, they process, sell, and re-sell dataavailable on the Web. By doing that, these plat-forms can provide added value in numerous ways.First, some data may be hard to find and scatteredacross numerous websites. A data vendor that ag-gregates these single datasets into a bigger and morerefined one performs a service that makes it easier forcustomers or end-users to access relevant data. Sec-ondly, datasets from different providers often havedifferent access mechanisms and formats. There-fore, offering one single mechanism to access datain a consistent format can save time and money forcustomers.

This has also been realized by information provi-ders who seek commercialization of their data. Inaccordance with that, it can be observed that ev-ermore suppliers of data emerge. Aggregating andcurating this data into accessible and understand-able datasets is a business opportunity with highpotential, driven by the over-supply of data.

While there have been small, not primarily scien-tific surveys of data marketplaces ([7, 10, 11]) andresearch on specific data marketplaces such as theWindows Azure Marketplace [9] and others (e. g.,[12]), there is—to our knowledge—to date no com-prehensive survey and comparison of multiple datamarketplaces and data vendors. Therefore, we haveconducted such a survey, including a total of 46 sup-pliers of data. The study was conducted from Aprilto July 20121 with the aim of identifying categoriesand dimensions of data marketplaces as well as ven-dors of data in order to build a taxonomy for datamarketplaces.

1The list of companies surveyed can be foundat http://dbis-group.uni-muenster.de/temporary_downloads/SurveyList.pdf, and we are happy to pro-vide the full data of the survey upon request. However,because data marketplaces are a very vivid field andchange fast, it has to be pointed out that Kasabi wentout of business since the survey was taken.

SIGMOD Record, March 2013 (Vol. 42, No. 1) 15

Surveying the current state of affairs in this fieldcan be seen as the first step in analyzing and un-derstanding this emerging market. We plan on re-peating this study annually in order to gain furtherinsight about what has changed, which competitorshave been successful or not and why, which modelsand practices have proven themselves, etc. Research-ing the market and its developments can not onlyhelp understanding the market dynamics but alsocan give valuable insights into the emergence or ap-plication of new technologies and, thus, present newresearch opportunities.

The remainder of this paper is organized as fol-lows: First, the survey approach will be described inSection 2. Then we present our findings, i. e., group-ings and categorizations in Section 3. Section 4 givesan overview of related work that has been conductedin this area. The paper is concluded by summarizingour findings in Section 5.

2. METHODOLOGY AND APPROACHIn this section, we first elaborate on what we

consider to be a data market or data vendor. Thenwe explain how the survey was conducted, using aniterative approach for both collecting data suppliersand deriving categories in Section 2.2. Section 2.3discusses limitations of the method applied.

2.1 Data Marketplaces and Data VendorsIn the context of this work we have analyzed data

vendors and data marketplaces. In order to restrictthe potentially vast amount of companies, we havefocused on companies offering either a platform fortrading data (e. g., datamarket.com), raw data inany form (e. g., www.data.gov), or data enrichmenttools (e. g., attensity.com). In order to gain a com-parable set of data vendors, we have chosen to focuson vendors that offer online Web services. This im-plies that we have excluded offline products for datacleansing or data fusion and similar tasks.

We define a data marketplace as a platform onwhich anybody (or at least a great number of po-tentially registered clients) can upload and maintaindata sets. Access to and use of the data is regulatedthrough varying licensing models.

A data vendor has data and offers it to others,either for a given fee or free of charge. However,it is not important how vendors obtain this data,and many ways are common, e. g., aggregation fromfreely available sources, generation using proprietarymethods, or buying from other vendors. It is impor-tant to note that a data vendor can offer its dataeither on its own or through a data marketplaceas described above. Conversely, it is also possible

that a data marketplace operator sells data and thustakes on the role of a vendor.

In our understanding, data marketplaces and datavendors have evolved from traditional Web crawlersand search engines as they all provide users withdata. That is why we chose to also include crawlersand search engines that were comparable. Addition-ally, we also looked at data enrichment services thattake input from the user and enhance it in someway, e. g., by analyzing or tagging it. Seeing howthese services face the same data curation challengesas data marketplaces do, we allowed them into thissurvey.

2.2 Data Acquisition and ApproachThe initial set of vendors consisted of well-known

suppliers we found in previous research [14]. Fromthis starting point, keywords were derived that werethen used for a broader online search, which in turnrevealed a more comprehensive set of different prod-ucts and services.

We came up with a set of twelve dimensions alongwhich the vendors considered can be categorized.As not all dimensions are measureable, and the di-mensions are grouped into objective and subjectivedimensions to clarify where our own opinion hasinfluenced the results. Table 1 shows the dimen-sions that we used, the categories that constitutethis dimension as well as the questions we asked toconduct this survey.

The values in our approach are strictly Boolean.An offering either fulfills the criteria for a certaindimension category or it does not. However, cate-gories are not mutually exclusive in most cases. Thismeans that, e. g., one offering can fall into multiplecategories, have multiple pricing models, or providemultiple ways for data access. Some dimensions(e. g., maturity), however, are mutually exclusive.Where this is the case, it will be stated explicitly inthe dimension description in Section 3.

The facts about the data vendors were gatheredby means of a Web search. As every vendor ormarketplace has a website, this publicly availableinformation was used to determine how to categorizeeach vendor. After having done that with the initialset of vendors, it was checked how many entries acategory had to justify its existence. When a cat-egory had only few entries, a new Web search formore data suppliers falling into that category wasstarted in order to make sure no important vendorswere omitted. If more companies were found, thelist was extended iteratively, and the new companieswere analyzed regarding the other dimensions. How-

16 SIGMOD Record, March 2013 (Vol. 42, No. 1)

Table 1: Set of dimensions.

Dimension Categories Question to be answered

obje

ctiv

e

Type Web Crawler, Customizable Crawler, SearchEngine, Pure Data Vendor, Complex Data Vendor,Matching Vendor, Enrichment Tagging,Enrichment Sentiment, Enrichment Analysis, DataMarket Place

What is the type of the core offering?

Time Frame Static/Factual, Up To Date Is the data static or real-time?Domain All, Finance/Economy, Bio Medicine, Social Media,

Geo Data, Address DataWhat is the data about?

Data Origin Internet, Self-Generated, User, Community,Government, Authority

Where does the data come from? Who is the author?

Pricing Model Free, Freemium, Pay-Per-Use, Flat Rate Is the offer free, pay-per-use or usable with a flat rate?Data Access API, Download, Specialized Software, Web

InterfaceWhat technical means are offered to access the data?

Data Output XML, CSV/XLS, JSON, RDF, Report In what way is the data formatted for the user?Language English, German, More What is the language of the website? Does it differ

from the language of the data?Target Audience Business, Customer Towards whom is the product geared?

subje

ctiv

e Trustworthiness Low, Medium, High How trustworthy is the vendor? Can the original datasource be tracked or verified?

Size of Vendor Startup, Medium, Big, Global Player How big is the vendor?Maturity Research Project, Beta, Medium, High Is the product still in beta or already established?

ever, if no more companies were found, the categorydefinitions were reconsidered and updated.

2.3 LimitationsThe information we used was taken directly from

the website of each vendor. This may limit theaccuracy of our findings in some cases, where thedescription of a product exceeds the actual function-ality. Verifying that every product fulfills its owndescription is a task that goes beyond the purposeof this survey. Random samples, however, indicatethat the descriptions commonly match the servicesprovided. Nevertheless, there are also cases wherethe information provided on a vendor’s website wasnot sufficient to categorize all dimensions. This wasparticularly the case for B2B vendors, which only re-veal their pricing models upon request. We chose toleave these dimensions out than to speculate abouttheir value. As a result, however, the numbers ofthese dimensions are minimally skewed.

The market of data vendors and data marketplaces is highly active, i. e., new actors emerge andothers disappear, and the market as such is growingrapidly. Therefore, it cannot be guaranteed that thisstudy is fully exhaustive with regard to the numberof vendors in the market. That said, we are confidentthat during our observation period from April toJuly 2012 we have obtained a representative samplethat allows for a meaningful analysis. Furthermore,it has to be stated that data trading channels arenot necessarily made public. This means that weare aware of the fact that a certain amount of datais traded directly between (large) corporations or

within a certain ecosystem (such as social networks)without the use of intermediaries. It is obvious thatit is impossible to investigate those forms of datatrading using our Web survey approach.

3. FINDINGSAs stated in the previous section, the following

twelve dimensions have been examined: Type, TimeFrame, Domain, Data Origin, Pricing Model, DataAccess, Data Output, Language, Target Audience,Trustworthiness, Size of Vendor, and Maturity. Tostructure these dimension we have categorized theminto objective and subjective measures, i. e., whetherthe classification within each dimension can be easilyverified or whether the classification is down to theresearcher’s judgement.

3.1 Objective Dimensions

3.1.1 TypeThe first dimension type is used to classify vendors

based on what their core product is. In order to forma common understanding of the different categoriesthese are explained below:

• (Focused) Web Crawler: Services that are specif-ically designed to crawl a particular websiteor set of websites. These are always bound toone domain, e. g., spinn3r is a service that isspecialized on indexing the blogosphere.

• Customizable Crawler: General purpose craw-lers that can be set up by the customer to crawl


0

5

10

15

20

Figure 1: Number of vendors for each Type.

any website and search for arbitrary content.For example, 80legs offers such a service, inwhich customers can define regular expressionsto crawl a set of sites.

• Search Engine: Services that offer their con-tent via an interface similar to a search engine.Customers specify combinations of keywordsas input and the search engine produces out-put relevant to that input. FactForge is sucha search engine that represents an interface tothe Linking Open Data cloud.

• Raw Data Vendor: This category comprisesvendors that offer raw data, most often in theform of tables or lists. For example, Factualoffers lists of restaurants, hotels, and otherpoints of interest.

• Complex Data Vendor: These vendors offerdata that is the result of some kind of analysisprocess. For example, The Stock Sonar pro-vides information about current stock pricesas well as indicators on how individual sharesmight develop in the near future.

• Matching Data Vendor: Vendors that offer thematching of input data against some other data-base. These vendors most often operate indomains where a customer does not want acomplete dataset, but rather needs the datathey already have corrected or verified, e. g.,address data. Companies like AddressDoctorare specialized in this area.

• Enrichment – Tagging: This category describesservices that enrich a given input (mostly text,but other forms are also possible) throughmeans of tags. This enables customers to make

more use of their data. Calais for examplecreates metadata for content submitted usingnatural language processing.

• Enrichment – Sentiment: With the prolifera-tion of social media websites on the internet,a multitude of vendors has emerged that spe-cialized on what is commonly referred to assentiment analysis [15]. Given the name ofa brand or a product, these services try tocapture and analyze the sentiment of peopletowards that subject. This kind of service is,for example, offered by Salesforce under thename Radian6.

• Enrichment – Analysis: The data offered isenriched with analysis results obtained throughvarious means, like comparisons with historicaldata or forecasts. Attensity Analyze is oneof such services, offering customer analyticsacross multiple channels.

• Data Market Place: These services allow cus-tomers to both buy and sell data by providingthe infrastructure needed for such transactions.A prime example for this type of vendor isMicrosoft’s Windows Azure Marketplace.

Figure 1 shows how many vendors fall into whichcategory. It has to be kept in mind, though, thatthese categories are not mutually exclusive and onevendor can fulfill the criteria of multiple categories.Also, it should be noted that this histogram onlyshows a distribution over our sample and does notrepresent the entire market. This is owing to the factthat (as stated in the Section 2) we have intentionallyexcluded offline providers and tools.


3.1.2 Time FrameThe time frame dimension captures the temporal

context of the data. We distinguish two categoriesin this dimension:

• Static/Factual: Data is valid and relevant fora long period of time and does not changeabruptly, i. e., population numbers, geographi-cal coordinates, etc.

• Up To Date: Data is important shortly afterits creation and loses its relevance quickly, i. e.,current stock prices, weather data, or socialmedia entries.

As evident from Figure 2, we found that staticdata (32 offerings) was offered more often than up-to-date data (23 offerings). Some vendors offer datafrom both these categories. For example, Data.govoffers real-time data about worldwide earthquakesfor the past 7 days as well as a dataset containinginformation on the total calories of commonly eatenfoods. However, we found that only less than 20%(9 offerings) of the surveyed vendors offer both staticand up to date information. This suggests thatgenerally data vendors tend to specialize in eitherof the two options.

0

5

10

15

20

25

30

35

Static/Factual

Up ToDate

Figure 2: Number of vendors for TimeFrame.

3.1.3 DomainThe dimension domain describes what the actual

data is about. While most domain names are self-explanatory, domain any deserves clarification. Thisdomain was used to classify vendors whose offersare not restricted and could incorporate arbitrarydomains. For example, the Windows Azure Mar-ketplace is not focused on a specific domain, whichmeans that all different kinds of data can be foundthere. Whilst other domains were not mutually ex-clusive (i. e., a vendor could supply more than onedomain), vendors serving any domain did not count

towards explicit domains. The results are shown inFigure 3.

It is obvious that the any domain is by far thebiggest group. An explanation for this is that datamarket places, search engines, and customizablecrawlers do indeed serve any domain, dependingon what customers choose to upload or search for.Given that they account for more than a fourthof all companies under investigation, the peak inany is not surprising. The other domains have alower number of vendors, because they are morespecialized. Furthermore, we have observed thatthe geo data (7) and address data (8) domains havea significant overlap (6), which can be explainedby their obviously close relationship. Companieslike AggData specialize in providing high-qualitydata about customers and their locations, so theyfit into both categories. Address and geo data are,however, not the same, as evidenced for exampleby CustomLists.net, who offer only address data formarketing purposes.

3.1.4 Data OriginThe origin of data describes where it comes from.

We have identified six different categories in thisdimension:

• Internet: The data is pulled directly from apublicly and freely available online resource.

• Self-Generated: Vendors have means of gener-ating data on their own, i. e., manual curationof a specific dataset or calculating forecastsbased on patented methods.

• User: Users have to provide an input beforethey can obtain any data, i. e., address dataofferings that return the address for a givenname.

• Community: Based on a wiki-like principle,these vendors obtain and maintain their datain a very open fashion. The restrictions as towho can participate and contribute are usuallyrather low.

• Government: Governments capture and pro-cess huge amounts of data and have recentlybegun to make this data publicly available.

• Authority: Authorities in a domain are enti-ties which are the main provider of data, i. e.,the stock market for stock prices or the postaloffices for address data.

In our survey the most popular origin categorywas the Internet. Almost 50% of all vendors receivetheir data from an online source. Another category


0

5

10

15

20

25

30

Any Finance/Economy

BioMedicine

SocialMedia

Geo Data AddressData

Figure 3: Number of vendors for each Domain.

0%5%

10%15%20%25%30%35%40%45%50%

Figure 4: Data origin distribution.

with a large number of vendors was authority : 32%obtain their data from authoritative sources. Forexample, Intelligent Search Technology claims thattheir address verification service is certified by theU.S. Postal Service. The main advantage of theseoffers is that the data is usually of high correctness,completeness, and credibility. This also holds forthe government category, into which fell 15% of ven-dors. The categories self-generated and communityare matched by 15% and 19%, respectively. Theproblem with self-generated data is that there is notransparency in the data sourcing process. For ex-ample, CustomLists.net does not reveal where theyget their data from, which might raise concerns re-garding credibility or correctness. Lastly, categoryuser with 15% is a special case because it cannotstand on its own, i. e., every vendor classified intothis category also gathered data from another source.This is inherent to the definition of this category,according to which users submit their data and re-ceive it back with additional annotations for whicha vendor needs additional data sources. These factsare illustrated in Figure 4.

3.1.5 Pricing ModelPricing models are very important to understand-

ing how exactly the different vendors set up theirbusiness models. Four main pricing models couldbe found; the number of vendors for each model isillustrated in Figure 5. A verbal explanation of thepricing models is provided by the following list:

• Free: These services can be used at no charge.Reasons for offering a service for free are, amongothers, that it is only a beta test or researchproject, the vendor is a public authority fundedby tax money, or simply interested in attract-ing more customers. For example, Data.gov isfree as it is a website of the U.S. government.Vendors in this category do not count towardsone of the following categories.

• Freemium: As a portmanteau combining freeand premium, this pricing model offers a lim-ited access at no cost with the possibility of anupdate to a fee-based premium access. Free-mium models are always combined with at leastone of the following two payment models.


• Pay-Per-Use: Customers are billed based onhow much they use the respective service. Thismanifests mostly in the form of x$ per thousandAPI calls.

• Flat Rate: After paying a fixed amount ofmoney, customers can make unlimited use ofthe service for a limited time, mostly a monthor a year.

02468

10121416

Figure 5: Number of vendors for each Pric-ing Model.

An example for the combination of the Freemiumand Pay-Per-Use model is Factual.com. Their APImay be called up 10,000 times per day for free. Anyadditional calls have to be paid for. The CloudMadeData Market Place, on the other hand, combinesFreemium with Flat Rates by offering free trials fortheir datasets and unlimited access for an annualfee.

3.1.6 Data AccessThe data access dimension describes through which

means end-users receive their data from vendors.The main categories identified and presented in Fig-ure 6 are:

• API: An API (application programming in-terface) is used to provide a language- andplatform-independent programmatic access todata over the Internet.

• Download: Traditional download of files is theeasiest way to access a data set, because anyonecan use such a service with only a Web browser.

• Specialized Software: Some vendors have im-plemented a specialized software client to con-nect with their Web service. While this ap-proach does have downsides (implementationand maintenance expense, dependency issues,etc.), there are some scenarios in which the con-cept is worthwhile, for example, providing thecustomer with an easy-to-use graphical user in-terface as an out-of-the-box solution that needs

no further customization, or granting access toreal-time streams of data.

• Web Interface: In a Web interface, the data isdisplayed to the customer directly on a website.

The flexibility and modularity of APIs have madethese the most popular of all access methods. Morethan 70% of all vendors offer an API. However, lessthan 30% of all vendors have an API as their onlyway to access data. Most vendors offer an APInext to other methods. For example, Web interfacesor file downloads are used to give previews of thedataset, to make it easier and more accessible for thecustomer to see what the actual data looks like, e. g.,Factual.com has an extensive Web frontend that ren-ders tables or geodata. The concept of specializedsoftware does not seem to stand very well on itsown. Out of all investigated vendors, only three usespecialized software as the only way of data access.For example, MeaningMine provides the user witha dashboard-like interface that shows graphs andimportant numbers. However, this approach lacksflexibility, because customers are restricted in theway they can use the data by the functionality ofthe provided software. Nevertheless, most customerswho want data do not want any restrictions on howthey can access and process the data. From a theo-retical point of view, it seems to be the best approachfor a vendor to offer all the aforementioned meansof access to his data, because that allows customersto choose their preferred way of access. However, wehave not found a single vendor that does so, whichis probably due to the high cost associated withcreating such a broad offering.

3.1.7 Data OutputThis dimension shows the format in which data

can be obtained. To us, the most reasonable set ofcategories in this dimension is the following:

• XML: Being both human- and machine-read-able, the Extensible Markup Language is awidely established standard for data transferand representation.

• CSV/XLS: Most structured data is laid outin a tabular way, so it makes sense to wrap itinto a table file format. We do not distinguishbetween CSV and XLS and other table fileformats, because the main differences betweenthem, like formatting and embedding, do notapply when you are showing raw data

• JSON: The JavaScript Object Notation is sim-ilar to XML and is also used as a data transfer


0%

10%

20%

30%

40%

50%

60%

70%

80%

API Download SpecializedSoftware

Web Interface

Figure 6: Data Access distribution.

0

5

10

15

20

25

XML CSV/ XLS JSON RDF Report

Figure 7: Number of vendors per Data Output category.

format. Data is represented as text in key-values pairs.

• RDF: The Resource Description Framework isa method to describe and model information. Ituses subject-predicate-object triplets to makestatements about resources. Due to its graphdata model, it is a good choice for data that isinherently graph-shaped.

• Report: When data is preprocessed, aggregatedand “prettified” in some way, we declared theoutput as a report. The main difference in thiscategory is that the customer does not have in-sight into the underlying raw data. Also visualreports in the form of MS Excel spreadsheetclassified for this category.

The most popular category in the output dimen-sion shown in Figure 7 is CSV/XLS. With 22 ven-dors, almost half of all vendors considered offer thepossibility to receive their data as a raw table. How-ever, only six of those vendors have CSV/XLS astheir only output format. Most vendors also of-fer either an XML (10) or a JSON (6) interface,some even both (3). This is consistent with the

observation from the previous dimension, that anAPI is the most popular way of data access. AnAPI usually produces XML or JSON output. Offer-ing many ways to access data is a key feature of adata marketplace, because it broadens the range ofpossible users. DataMarket.com therefore supportsall aforementioned output categories except RDF.Other competitors, however, do not provide all thesedifferent access mechanisms. The Infochimps DataMarketplace favors JSON over XML for their API.It remains to be seen what further implications thistechnical limitation may have.

3.1.8 LanguageWe have focused on the English and German lan-

guages because of personal language skills. Thus,further differentiations in this dimension were notpossible. Therefore, any additional languages weencountered were aggregated into a third categorycalled more. Although English is a dominant lan-guage on the Internet, we would be happy to cooper-ate with other researchers with other language skillsin a future edition of the survey.

The analysis of language distinguishes betweenthe language of the website and the language of the


0%

20%

40%

60%

80%

100%

EN DE More

0%

20%

40%

60%

80%

100%

EN DE More

Figure 8: Language of websites (left) and data (right).

data offered. A visual representation of the resultsis shown in Figure 8. Nearly all investigated vendors(98%) run an English-language website. For themajority, English is also the only language available(89%). Only some companies run a multilingual web-site (9% German; 7% More). These tend to be thebigger player with a global strategy, like Microsoftor LexisNexis. This picture changes when lookingat the language of the data itself. We observed thatagain 98% offered English Language Data, but about30% offered German data and almost 20% of thevendors also offered data in other languages.

We have seen that English is the dominant lan-guage for both websites and data. This is not sur-prising because the market for data has a globalscope and English seems to be the best suited lan-guage for that. However, there is also a demand forlocal data in the corresponding language, which issuggested by the amount of vendors that offer suchdata.

3.1.9 Target AudienceThe last objective dimension is concerned with

the target audience. Here, we have investigated to-wards whom offerings are tailored. As is evidentfrom Figure 9, there are only two categories in thisdimension, business and customer. Providing datafor another company in a B2B fashion is the mostlogical application area of data vending. Specializedvendors focus on their respective domain, e. g., Cus-tomLists.net targets business users while WolframAlpha is aimed more at private users. The moregeneral vendors, especially those operating in theany domain like Kasabi or Windows Azure Mar-ketplace, target their offer at all audiences. Outof all vendors in this research, 87% offered data ina business context, 41% sold data relevant for endconsumers, and 28% had data that could be of usefor both groups.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Business Customer

Figure 9: Number of vendors by Target Au-dience.

3.2 Subjective Dimensions

3.2.1 TrustworthinessThis dimension indicates how trustworthy the

data of a vendor is, depending on the origin of thedata as well as on how it is processed. For instance,data that come from a community could have alower trustworthiness than data that is sourced froman authority. In other words, data from a postaloperator as offered by, e. g., AddressDoctor is morelikely to be correct than an aggregation of onlinesources. However, there are also other cases where acollective of anonymous authors produce data thatis verifiably correct and therefore trustworthy, e. g.,Wikipedia. Whether more trust is put in a singleauthority of a domain or in a crowd of people de-pends on the application context and one’s personalattitude. Nevertheless, this dimension is not quan-tifiable and, thus, the results are subjectively biased.

As depicted in Figure 10, we have found that 54%of all vendors have a high trustworthiness. Amongthose are vendors that carefully select the data theyoffer in a transparent and comprehensible way. Also,authorities and governments as explained in Sec-tion 3.1.4 all exhibit a high trustworthiness. Thecategory medium is populated by around 33% ofall examined vendors. The main indicator for their


classification that they seem to be trustworthy isbased on the descriptions, but this could not be ver-ified in any way, e. g., because they do not explicitlystate their data sources or explain their analyticalmethods. The lowest degree of trustworthiness ap-plied to only 22% of all vendors. Typical vendorsin this category are those that do not even claim todeliver correct or complete data, like web crawlers(e. g., 80legs) or community-supplied websites (e. g.,Freebase).

0%

10%

20%

30%

40%

50%

60%

Low Medium High

Figure 10: Trustworthiness distribution.

Note that the overlap between the three categoriesstems from the fact that one vendor can offer mul-tiple datasets from different sources, like Kasabi orInfochimps. In such a case, we have assigned all pos-sible levels of trustworthiness. Furthermore, whileit is intuitive that high trustworthiness is good, it isnot necessarily the case that a low trustworthiness isbad. There are scenarios in which incomplete datais sufficient for a rough estimation, or data witha high trustworthiness is not available (e. g., socialmedia analysis). This leads us to the conclusionthat vendors with all levels of trustworthiness arelikely to co-exist in the future, because they fulfilldifferent demands.

3.2.2 Size of VendorWhile some might argue that the size of a vendor

is quantifiable (e. g., using the number of employeesor its revenue), and thus, an objective dimension, itis difficult to find reliable figures that would supportsuch an analysis. We therefore took the presentationof the offering as a foundation for a classificationwith the following four categories:

• Startup: Companies that are newly createdand that have only a small number of peopleinvolved are usually referred to as startups; ex-amples include Uberblic or QuantBench. Theseare often funded by investors, as they do notyet have a positive cash flow from the verybeginning.

• Medium: Leaving the beta stage, gaining expe-rience and maturity, and not being dependenton investors anymore are the key character-istics that set medium-sized companies apartfrom startups. Examples include eXelate orSpinn3r.

• Big: Companies that are well-established andhave more than one product in their offeringrange are considered big, e. g., Infochimps orLexisNexis. While there is no sharp dividingline between medium-sized and big companies,we still felt that separating the two in differentgroups yields more accuracy for the analysis.

• Global Player: In this category fell only thebiggest companies out there, like Yahoo!, Mi-crosoft, IBM, etc.

0

2

4

6

8

10

12

14

16

18

Startup Medium Big GlobalPlayer

Figure 11: Number of vendors by size.

Note that in this dimension, the categories aremutually exclusive. Figure 11 shows the numberof vendors for each size. It can be seen that thenumber of startups is the lowest. This could indicatethat the market for data is not easy to enter. Thenumber of global players also seems rather low, butit has to be kept in mind that these vendors havethe potential to quickly seize huge market shares,because they usually have experienced people andextensive capital. The majority of vendors is eithermedium-sized or big.

3.2.3 MaturityThe maturity of all offerings has been classified

into the following four categories, which are mutuallyexclusive:

• Research Project: These offerings are usuallynot for profit and can therefore be used free ofcharge. They are mainly executed as a proof-of-concept. Examples include Goolap or IBMCognos Many Eyes.


• Beta: A beta product is still in developmentand has not been fully launched yet. Neverthe-less, we have also seen offerings in beta phasethat already demanded a usage-fee, like Seman-tics3.

• Medium: This category classified products thatwere already out of beta, but were still not ashighly developed as other products, such asBuzzData or CloudMade Data Market Place.

• High: Full-fledged products that implementall intended features and are ready for use inan operational environment. For example, theWindows Azure Marketplace seems to be rela-tively advanced in this sense.

0

5

10

15

20

25

30

35

ResearchProject

Beta Medium High

Figure 12: Maturity of vendors.

Evaluating the numbers presented in Figure 12shows that only 3 research projects, 6 betas and 6medium-matured offerings could be identified. Theremaining 31 offerings can all be classified as having ahigh maturity. This observation can also serve as anexplanation to the previous finding of a low numberof startups. When there are already establishedvendors with mature projects, the space for newcompanies to enter the market is relatively small.

4. RELATED WORKIn [6] a general discussion of data services can be

found. Starting from a general data service archi-tecture, the authors examine concepts and exampleproducts for service-enabling data stores, integrateddata services, and cloud data services. Further, theyhighlight technical challenges, such as transactionsan updates to data structures underlying the ser-vice, as well as predict emerging trends, such asconvergence and cloud integration.

Ge et al. [8] studied electronic marketplaces butrestricted themselves to Web sites where users canask questions (e. g., Askjeeves.com), which are thenanswered by other users or experts. Furthermore,they only described five websites and focused rather

on business models than on surveying marketplaceproperties.

Regarding data markets as we defined them inSection 2.1 surveys have only been done on a (much)smaller scale, not disclosing any methodology, andonly in textual form. For instance, Strata [7] describecharacteristics of the four (according to them) mostmature data markets Factual, Infochimps, DataMar-ket, and Windows Azure Data Marketplace, whichwe also examined in this study.

Similarly, Miller interviewed 10 providers of datamarketplaces or data related services in a series ofPodcasts [10]. However, he only provides the inter-views in a rather unprocessed form, i. e., as audiofiles, which makes it difficult to access and aggregatethe contained information. Later, he published areport [11] on data marketplaces and their businessmodels, in which he identified common function-alities that data marketplaces offer, elaborated onpotential business models and makes some rathergeneral predictions such as increasing competitionand a wider choice of data and sources.

Furthermore, there have been investigations intoparticular market places, for instance on Kasabi [12],which went out of business in the meantime. Itwas described as a ”web-based information market-place” and stored data using the Resource Descrip-tion Framework (RDF) with the goal of bridgingthe gap between data publishers and applicationdevelopers by providing a platform that allows host-ing of and searching for data. It was designed afterthe linked data paradigm originally outlined by TimBerners-Lee. The basic idea of linked data is topublish data in a structured way that allows forlinkage to data sets. An overview of this concept,the technical principles and its applications can befound in [4]. A survey about the current usage ofthese dataset is given by [13] and actual trends areoutlined in [3].

In the course of the Linked Open Data (LOD)movement, FactForge emerged as a publicly availableservice that is meant to ”provide an easy point ofentry for would-be consumers of Linked Data” [2].It was built with the intention to facilitate accessto the LOD cloud of data by integrating the majordatasets into one view.

A different approach is pursued by the developersof Freebase. They try to create what they call a”collaboratively created graph database for structur-ing human knowledge” [5]. The collaboration aspectis inspired by Wikipedia and based on the idea thatdata quality improves when lots of people refinedatasets. They employ a graph database, because itdepends less on a rigid schema and is more flexible.


The authors state explicitly that they want to allowconflicting and contradictory types and propertiesto exist simultaneously in order to ”reflect users’differing opinions and understanding” [5].

DBPedia is a different project that shares manysimilarities with Freebase. They both aim at ex-tracting structured data and making it available inRDF. However, DBPedia focuses on Wikipedia as itsonly source, and also does not allow direct editingof data.

Microsoft’s contribution to the market is WindowsAzure Marketplace [9] and has been launched in 2010.It is designed to make the sharing of data as well asapplications an easy process for both consumers andproviders of data. The key features are global reachthrough a central platform, unified billing and accessmechanism, high data quality, and easy integrationwith other Microsoft products. Unique to WindowsAzure Marketplace is the way in which datasetsand applications are combined. Providers of datacan go beyond selling their raw data, and bundle itwith applications that are designed specifically forthis dataset. Customers can purchase these bundlesdirectly and have a working out-of-the-box solutionwithout any additional implementation effort.

That said, there is—to our knowledge—no sur-vey that investigates data marketplaces in such acomprehensive manner as we have done.

5. CONCLUSION & FUTURE WORKIn this study we have presented an initial overview

of data vendors and marketplaces for data. Utilizingan iterative approach we have derived dimensionsalong which data providers can be classified andgrouped. We have then presented a survey drawinga preliminary picture of the current data vendorlandscape. Our survey gives an overview of the cur-rent market situation and shows which categories arecurrently underrepresented and which ones can beparticularly interesting for practitioners. However,it is too early to make reliable statements aboutwhere data marketplaces are heading. That is whywe plan on repeating this survey on an annual basisto re-evaluate the individual vendors and extendingthe study with a development section. We believethat a comparison over time will allow for assessingwhich models and practices stand the test of time.Also, technical trends can then be deduced frommarket observations and give valuable insights toresearchers.

This study has focused on the provider view ofdata marketplaces, which have emerged because ithas by now been recognized that and how datacan be monetized. It will also be interesting to

observe buyers of data and analyze their perceptionof these new offerings, where a distinction betweenprivate and professional customers is likely to beappropriate. Here, it will over time be possible todetermine who is spending money for what kindof data, and we expect that certain domains willbe more attractive than others. For example, avariety of current activities in the healthcare domain(e.g., taltioni.fi, ensembl.org, patientslikeme.com, orcancerresearchuk.org) indicates a high attractivenessfor data markets. This will be a subject of futureresearch.

AcknowledgmentWe like to thank the anonymous reviewers of the ACMSIGMOD Record for their helpful comments.

6. REFERENCES[1] A.A. Armstrong and E.H. Durfee. Mixing and memory:

emergent cooperation in an information marketplace. InMulti Agent Systems, 1998. Proceedings. InternationalConference on, pages 34 –41, jul 1998.

[2] B. Bishop, A. Kiryakov, D. Ognyanov, I. Peikov,Z. Tashev, and R. Velkov. FactForge: a fast track to theweb of data. Semantic Web, 2(2):157–166, April 2011.

[3] C. Bizer. The Emerging Web of Linked Data. IEEEIntelligent Systems, 24(5):87–92, 2009.

[4] C. Bizer, T. Heath, and T. Berners-Lee. Linked Data -The Story So Far. International Journal on SemanticWeb and Information Systems (IJSWIS), 5(3):1–2,2009.

[5] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, andJ. Taylor. Freebase: a collaboratively created graphdatabase for structuring human knowledge. In SIGMOD’08: Proceedings of the 2008 ACM SIGMODinternational conference on Management of data, pages1247–1250, New York, NY, USA, 2008. ACM.

[6] Michael J. Carey, Nicola Onose, and MichalisPetropoulos. Data services. Commun. ACM,55(6):86–97, June 2012.

[7] Edd Dumbill, 2012. http://strata.oreilly.com/2012/03/data-markets-survey.html.

[8] W. Ge, M. Rothenberger, and E. Chen. A Model for anElectronic Information Marketplace. AustralasianJournal of Information Systems, 13(1), 2005.

[9] Microsoft White Paper. Windows Azure Marketplace,2011. http://go.microsoft.com/fwlink/?LinkID=201129&clcid=0x409.

[10] Paul Miller, 2012. http://cloudofdata.com/category/podcast/data-market-chat/.

[11] Paul Miller, 2012. http://pro.gigaom.com/2012/08/data-markets-in-search-of-new-business-models/.

[12] K. Moller and L. Dodds. The Kasabi InformationMarketplace. In 21nd World Wide Web Conference,Lyon, France, 2012.

[13] K. Moller, M. Hausenblas, R. Cyganiak, andS. Handschuh. Learning from Linked Open Data Usage:Patterns & Metrics. In Web Science Conference, 2010.

[14] A. Muschalle, F. Stahl, Loser, and G. Vossen. PricingApproaches for Data Markets. In to appear in 6thInternational Workshop on Business Intelligence forthe Real Time Enterprise (BIRTE), 2012.

[15] B. Pang and L. Lee. Opinion Mining and SentimentAnalysis. Found. Trends Inf. Retr., 2(1-2):1–135,January 2008.


Documents

Marketplaces for Data: An Initial Survey - Plone sitemontesi/CBD/Articoli/MarketPlaceForData.pdf · any form ( e.g ., ), or data enrichment tools ( e.g ., attensity.com). In order