Ontology Guided Purposive News Retrieval and

Presentation

Abir NaskarTCS Innovation Lab, India

abir.naskar@tcs.com

Rupsa SahaTCS Innovation Lab, India

rupsa.s@tcs.com

Tirthankar DasguptaTCS Innovation Lab, Indiadasgupta.tirthankar@tcs.com

Lipika DeyTCS Innovation Lab, India

lipika.dey@tcs.com

Abstract

In this paper, we present a purposive Newsinformation retrieval and presentation systemthat curates information from News articlescollected from multiple trusted sources for agiven domain. A back-end domain ontologyprovides details about the concepts and re-lations of interest. We propose an attentionbased CNN-BiLSTM model to classify sen-tence tokens as ontology concepts or entities ofinterest. These entities are then curated andused to link articles to illustrate evolution ofevents over time and regions. Working sys-tems are initiated with small annotated datasets which are later augmented with humansin the loop. It is easily customizable for vari-ous domains.

1 Introduction

News consumption is no more restricted to consuminga set of facts dished out by a specific agency. Readersare not only choosing the type of content they want toread but also how. Increasing interest in social statis-tics is also seeing News as a source of data for gener-ating these statistics. Unlike social media, News fromtrusted sources is reliable. News presentation is there-fore undergoing a sea-change. Along with a bird’s eye

Copyright c© 2019 for the individual papers by the papers’ au-thors. Copying permitted for private and academic purposes.This volume is published and copyrighted by its editors.

In: A. Aker, D. Albakour, A. Barron-Cedeno, S. Dori-Hacohen,M. Martinez, J. Stray, S. Tippmann (eds.): Proceedings of theNewsIR’19 Workshop at SIGIR, Paris, France, 25-July-2019,published at http://ceur-ws.org

view of global events, the ability to delve deep downinto specific stories along various dimensions and alsowatch their evolution is necessary to enable systematicstudies.

In this paper, we present an Ontology-guided Newsinformation retrieval and presentation system. Theuniqueness of the proposed system lies in the use ofa back-end domain ontology that specifies the enti-ties and relations of interest in a domain, based onwhich, information components are extracted and clas-sified using a deep neural network architecture. Theseconcepts are used to create domain-specific ”purposiveindices” to aid concept-oriented Information retrievalrather than simple word-based retrieval.

A seed ontology of concepts along with a few in-stances of each concept is used to create annotateddata to train a concept classification model. This isapplied over a larger set of articles, the result of whichis then validated through human evaluation and usedsubsequently to enhance the initial model. It is foundthat the proposed method takes much less time and ef-fort to create annotated data sets for all situations thatlack large labeled data needed to exploit deep-learningmethods. Information components extracted from theNews articles, are stored in indexed repositories fordownstream analytics. The results are presented tothe end-user through an innovative interactive inter-face that helps in consuming information at multiplelevels of granularity.

2 Overview of proposed News Re-trieval and presentation framework

Figure 1 presents an overview of the proposed NewsInformation retrieval system. A number of Newscrawlers are deployed to collect News from dedicatedand reliable sources. For each article its meta-data like

SeedOntology

AnnotatedCorpus Train Classifier

& News Linking

CrawlerWeb

C1 C2 C3

ClassificationModelNews Docs

Entity AggregationInduced

Repository

News Retrieval

GoverningAgency

DueTo

occursAt

OnBy

propose

PenaltyViolation

Loc.

locatedAt

Incident

Safety

Target

Org

(A)

(B)

Crime

Accused Victim

Charges

Investigation

Law Enf

Commits Against

ResultsInAgainst

CarriesOutBrings

(C)Output

\Accused \Accused \None \None ... \Crime \Victim \Victim

Sanjeev khanna has been ........ killing Sheena Bora

EmbeddingsWord

r

a

CNNLayer

Bi−LSTM

21

a4

a

r1

aa3 a

n

2r rr

43

Linear layerWith sigmoid

Activation

nrr

n−2 n−1

an−1

n−2

w

AttentionPooling

Fully Connected

Figure 1: Overview of the Event and Entity Extraction Architecture(A). (B) depicts the example seed ontologystructure. (C) explains the CRNN architecture.

source, date and time of publication, location, head-line is stored along with the full article. These are allindexed using Solr. Exact or near duplicate articlesare identified using Latent Semantic Hashing (LSH).Though only one copy is stored in the repository, totalcount for such articles are maintained. Each articleis then passed through an information extraction andclassification pipe-line that deploys component classi-fiers to detect ontology-components in sentences. Theextracted elements are resolved and then used to cre-ate additional ”purposive named-indices” for the doc-uments. These elements are also used to link threadsof the same News together.

The ontology is composed of a schema that containsdifferent domain concepts (C) and a set of generic re-lations R between these concepts. Figure 1 illustratesa pair of ontologies for two different domains, namely,crime and occupational health and safety. Given anontology that elucidates the basic components of a do-main and their underlying relationships in a genericway, finding similar information components from vastcollections of text still remains a challenging task, sincethe manifestation of these components in natural lan-guage can be extremely varied. For example, thoughthe names of criminal and victim can be extracted asnamed-entities from text, establishing their roles un-ambiguously is not a simple task. Classifying an in-stance of a concept correctly requires deep contextualanalysis. We propose the use of an attention-poolingconvolutional Bi-LSTM neural network based archi-tecture to do the task. Details of this is given in thenext subsection.

Purposive indices for news articles are created us-ing the ontology concepts extracted from text. Forexample crime News articles are indexed by criminal

and victim names, section names, location of crimeetc. News articles reporting safety incidents are simi-larly additionally indexed by incident names, locationof incident, penalty incurred in different currency etc.Linking of related articles are done through the pur-posive indices only. It may be noted that all conceptsfor the same event may be obtained at once. Newconcepts can get associated as a single story unfoldsover time. Similarly multiple isolated articles may getlinked to each other at later stages. The purposiveindices are also used to generate comparative and ag-gregate statistics over various dimensions. The visu-alization module enables the end-user to view Newsarticles at various levels of granularity.

2.1 Ontology-guided Concept and Entity De-tection using C-RNN Network

Convolutional neural networks (CNN) exploit local de-pendencies, while recurrent networks like Long ShortTerm Memory (LSTM) capture long-distance depen-dencies among features. The proposed ConvolutionalBi-LSTM (C-RNN) model combines both the capa-bilities. For a given sentence, the network learns toassign ontology concept labels to each word. The in-put to the network is a sequence of word embeddingswith 100 dimensions each.

A convolutional layer is first used to extract local n-gram features. All word embeddings are concatenatedto form an embedding matrix M ∈ RdX|V |. Where,|V | is the vocabulary size and d is the embedding di-mension. The matrix is divided into k regions. In eachregion, we apply convolution function represented by

Conv( ¯xi:w) = W.( ¯xi:w) + b (1)

to calculate the output features. Where, W and b

are the weights that the network learns. We apply thesame convolution operation repeatedly over the dif-ferent matrix regions to get multiple output featurevectors. The output of the CNN layer is passed tothe bidirectional LSTMs, which read it both backwardand forward to take care of dependencies on the pastneighbours as well as future long-distance dependen-cies. The Bi-LSTM layer is followed by an attentionpooling layer over the sentence representations. At-tention modules have been proved to boost accuracyfor tasks like sentiment or activity detection by learn-ing to focus more on certain linguistic elements overothers, without increasing computational complexity.In our case, attention pooling achieves higher accu-racy by learning the specific characteristics surround-ing each concept in the form of weights associated tothe output of the Bi-LSTM layer. This is representedas:

ai = tanh(Wa.hi + ba), (2)

αi =ewα.ai∑ewα.ai

, (3)

O =∑

(αi.hi). (4)

Where Wa, wα are weight matrix and vector respec-tively, ba is the bias vector, ai is attention vector fori-th sentence, and αi is the attention weight of i-th sen-tence. The output of the attention layer is then passedto a fully connected linear layer with sigmoid activa-tion. The mapping of the linear layer after applyingthe sigmoid activation function is given by

y = s(x) = sigmoid(w.x+ b). (5)

Where, x is the input vector, w is the weight vector,and b is bias value. Finally, the loss function is com-puted using the cross-entropy loss defined by

L = −2∑i=1

yilog(yi). (6)

Where y is the one-hot representation of the actuallabel for the input word. To avoid over-fitting, weapply dropout technique at each layer to regularizeour model.

The model is initially built from a small anno-tated corpus, in which the instances of the ontologyconcepts are tagged by their respective labels. Thismodel, when applied over a larger corpus yields newinstances of each label, which are evaluated by hu-mans and then accepted for next-level training if twoout of three annotators simultaneously agree on thelabel. For repeated experiments on different domains,the inter-annotator agreement is found to be around0.65, which is pretty high. This can be done multipletimes, though we have restricted it to two times only.

As discussed earlier, classification of concepts ismore complex than merely identifying named entities.It is imperative to also recognize the role played bythe entity. Two such example sentences are presentedhere along with the concepts extracted -

Sanjeev Khanna has been taken into custody inKolkata on charges of killing Sheena Bora - Con-cepts extracted are <Criminal, Sanjeev Khanna> and<Victim, Sheena Bora>. Kolkata is not labeled as anyconcept, correctly.

US Department of Labor’s fines Heat Seal $95,000for 15 health violations Concepts extracted -<Company, Heat Seal> and <Penalty, $95,000>.

2.2 Linking Articles to Indicate News StoryEvolution

A link between two News articles is created only if theyshare purposive indices for specific concept classes. Forexample, for crime incidents, victim names and crimi-nal names should overlap, while for safety incidents or-ganization name and safety incident type should over-lap.

The first challenge comes in the form of entity reso-lution since named entities are spelled differently indifferent sources. An edit distance based measure[MV93] is used to compare different entities and com-bine them if sufficiently similar. For example, an indi-vidual was variably referred to as “Mukherjee”, “Muk-erjea” and “Mookerjee” across various sources in ourcrime news database.

The second challenge comes from the fact that thesets of conceptual instances for a single story alsoevolve over time. For example, it is observed that fora long-drawn crime incident, new articles report newnames as criminals or even victims, as new informa-tion pours in. A concept overlap threshold of 80% forspecified types is used to link the articles into a singlestory.

The third challenge is due to the fact that mereoverlap of the names of entities is not enough, eventheir corresponding roles need to be same, or at leastsimilar, for us to consider them to refer to the samecase.

Considering the above challenges, we propose aweighted similarity computation to determine the sim-ilarity of two articles. Highest weightage is given forcandidates that are resolved to be similar and also be-long to the same concept class. Candidates which aresimilar after resolution but are identified as instancesof different classes are given a lower weightage. The fi-nal similarity measure is computed as a weighted sumof all candidate similarities. Two articles are consid-ered similar if the similarity is above a user-definedthreshold.

Figure 2: Illustration of the linked News articles. Selecting one article shows the extracted events and entities.Graph at the top right shows the distribution of a News over time. Figure at the bottom right shows evolutionof entity types over time.

Once conceptually similar pairs of articles arefound, a virtual group is created containing these alongwith a union of the conceptual entities contained inthem. More articles are added to these by repeatingthe similarity computation of each article with an ex-isting group. The system automatically links themand maintains them chronologically.

2.3 News Retrieval and Data Visualization

Apart from regular entity and concept based retrievaland tracking the evolution of a News, the proposedsystem also enables the user to explore the evolutionof an incident across different dimensions like time andspace. Figure 2 illustrates how News evolution is pre-sented. The left most panel shows a series of crimeincidents reporting an unsolved murder case gatheredfrom across different time and sources. These havebeen linked together by the earlier algorithm using thepurposive indices extracted by the c RNN classifier.On selecting a particular crime news, the extracted en-tities and events are shown in the middle panel. Theright top chart shows the temporal distribution of thereports over the period 2015 to 2018. The bottomright chart displays how crime entities have changedover time. While Murder is prevalent over the en-tire time-line, new crime incidents like money launder-ing has emerged as reported co-crimes at later stages.One can also explore presence or absence of similarincidents across different geographical regions. It canunearth regional affinities for certain kinds of acts. Vi-sualizations also help in studying aberrations like dif-ferent charges evoked for similar crimes or variabilityin penalty rates for similar safety incidents throughcanned analytics.

3 Experiments and Results

3.1 Data Collection

We have conducted experiments for two different do-mains using the ontology pair described earlier. Ourcollection consists of around 12000 crime-related newscollected from the top 3 English news sources fromeach of four regions in the Indian subcontinent (north,south, east and west). The Occupational Healthand Safety database has been created from approxi-mately 4000 articles published by Occupational Safetyand Health Administration (OSHA), United StatesDepartment of Labor, each of which detail varioustransgressions by organizations, and the actions takenagainst them.

3.2 Experiments

Each dataset is divided into 70%, 20% and 10% fortraining, validation and testing respectively. A Con-dition Random Field (CRF) model is trained as thebaseline. This model uses part-of-speech(POS) andN-grams as features. Additionally, a number of otherdeep neural network based models such as BiLSTM,BILSTM with mean over time (MoT), BILSTM withAttention network, Convolution network (CNN), CNNwith BILSTM+MoT were used to compare the resultswith the proposed CNN with BILSTM along with At-tention Network. Each model is trained with threetypes of word embeddings GloVe(G), Word2Vec(w2v)and combination of GloVe and Word2Vec(G+w2v).Both w2v and a combination of Glove and w2v achievesimilar performance for the proposed architecture,which is significantly better than the baseline and alsoothers. F1 score for all the models are shown in Table1.

Table 1: Results demonstrating F1 Scores for each model corresponding to two domain, Crime and OSHACrime OSHA

G W2V G+W2V G W2V G+W2V

BiLSTM 67 70 64 64 65 70BiLSTM-MoT 66 69 68 63 66 72CNN 69 72 68 67 67 63BiLSTM-att 71 71 70 68 69 64CNN+BiLSTM-MoT 72 75 75 69 70 67CNN+BiLSTM-att 74 76 76 70 71 73

CRF 58 61

3.3 Hyper-parameters

For CNN, we keep the window size as 3 and number offilters as 30. For BiLSTM, state size is 200 with initialstate value =0.0. We use a dropout rate of 0.05, batchsize=10, initial learning rate of 0.01, decay rate of 0.05and gradiant clipping of 5.0.

3.4 Results

Throughout all the target classes, the performance ofthe CNN-BiLSTM model has been found to be betterthan the others. The performance of combined localand global embedding word2vec method for learningword embeddings [MCCD13] have been observed tobe very effective in capturing solely contextual infor-mation. It has also been observed that, combiningboth the W2V and GloVe embeddings surpasses theperformance of models using the individual embed-dings. Overall, the performance of the CNN-BiLSTM-att model along with combined W2V-GloVe embed-ding is higher than the rest of the existing models.

4 Related Works

While there has been a growing body of research in ex-tracting structured information from texts, neural net-work based ontology guided News event extraction andstory evolution is still in its nascent stage. Most of theexisting methods are limited to event and named entityextractions and not into identifying granular level ofentity role identification. Supervised learning with dif-ferent flavours of LSTM or CNN [GCW+16, MB16] areused for entity classification. Distant supervision in-volving some amount of annotated data and an initialknowledge source has been proposed to develop mod-els in [ZNL+09, NZRS12, CBK+10, MBSJ09, SSW09,NTW11]. The unsupervised approach requires handcrafted rules pertaining to the information to be ex-tracted [Hea92, SW13, JVSS98, HZW10, MB05].

5 Conclusion

In this paper we have proposed an Ontology-guidedNews information retrieval system using a Convo-lutional Bi-LSTM Network for Concept detection.

Concept-based linking is used to link related articles topresent News evolution and event distribution acrossregions. We have also illustrated how deep learningmethods can be deployed with small volumes of anno-tated data. We intend to extend the proposed methodsto work with all kinds of legal documents and also in-corporate predictive technologies to predict activitiesor events.

References

[CBK+10] Andrew Carlson, Justin Betteridge, BryanKisiel, Burr Settles, Estevam R Hr-uschka Jr, and Tom M Mitchell. Towardan architecture for never-ending languagelearning. In AAAI, volume 5, page 3. At-lanta, 2010.

[GCW+16] Jiang Guo, Wanxiang Che, Haifeng Wang,Ting Liu, and Jun Xu. A unified architec-ture for semantic role labeling and relationclassification. In Proceedings of COLING2016, the 26th International Conferenceon Computational Linguistics: TechnicalPapers, pages 1264–1274, 2016.

[Hea92] Marti A Hearst. Direction-based text in-terpretation as an information access re-finement. Text-based intelligent systems:current research and practice in informa-tion extraction and retrieval, pages 257–274, 1992.

[HZW10] Raphael Hoffmann, Congle Zhang, andDaniel S Weld. Learning 5000 relationalextractors. In Proceedings of the 48th An-nual Meeting of the Association for Com-putational Linguistics, pages 286–295. As-sociation for Computational Linguistics,2010.

[JVSS98] Yaochu Jin, Werner Von Seelen, and Bern-hard Sendhoff. An approach to rule-basedknowledge extraction. In Fuzzy SystemsProceedings, 1998. IEEE World Congresson Computational Intelligence., The 1998

IEEE International Conference on, vol-ume 2, pages 1188–1193. IEEE, 1998.

[MB05] Raymond J Mooney and Razvan Bunescu.Mining knowledge from text using infor-mation extraction. ACM SIGKDD explo-rations newsletter, 7(1):3–10, 2005.

[MB16] Makoto Miwa and Mohit Bansal. End-to-end relation extraction using lstms onsequences and tree structures. arXivpreprint arXiv:1601.00770, 2016.

[MBSJ09] Mike Mintz, Steven Bills, Rion Snow, andDan Jurafsky. Distant supervision forrelation extraction without labeled data.In Proceedings of the Joint Conferenceof the 47th Annual Meeting of the ACLand the 4th International Joint Confer-ence on Natural Language Processing ofthe AFNLP: Volume 2-Volume 2, pages1003–1011. Association for ComputationalLinguistics, 2009.

[MCCD13] Tomas Mikolov, Kai Chen, Greg Corrado,and Jeffrey Dean. Efficient estimationof word representations in vector space.arXiv preprint arXiv:1301.3781, 2013.

[MV93] Andres Marzal and Enrique Vidal. Com-putation of normalized edit distance andapplications. IEEE transactions on pat-tern analysis and machine intelligence,15(9):926–932, 1993.

[NTW11] Ndapandula Nakashole, Martin Theobald,and Gerhard Weikum. Scalable knowledgeharvesting with high precision and high re-call. In Proceedings of the fourth ACM in-ternational conference on Web search anddata mining, pages 227–236. ACM, 2011.

[NZRS12] Feng Niu, Ce Zhang, Christopher Re, andJude Shavlik. Elementary: Large-scaleknowledge-base construction via machinelearning and statistical inference. Interna-tional Journal on Semantic Web and In-formation Systems (IJSWIS), 8(3):42–73,2012.

[SSW09] Fabian M Suchanek, Mauro Sozio, andGerhard Weikum. Sofie: a self-organizingframework for information extraction. InProceedings of the 18th international con-ference on World wide web, pages 631–640. ACM, 2009.

[SW13] Fabian Suchanek and Gerhard Weikum.Knowledge harvesting in the big-data era.In Proceedings of the 2013 ACM SIGMODInternational Conference on Managementof Data, pages 933–938. ACM, 2013.

[ZNL+09] Jun Zhu, Zaiqing Nie, Xiaojiang Liu,Bo Zhang, and Ji-Rong Wen. Statsnow-ball: a statistical approach to extractingentity relationships. In Proceedings of the18th international conference on Worldwide web, pages 101–110. ACM, 2009.