Research ArticleStructural Analysis of Factual Conceptual Procedural andMetacognitive Knowledge in a MultidimensionalKnowledge Network

ETHurdica Vukic Sanda Martincic-Ipsic and Ana Mestrovic

University of Rijeka Department of Informatics Radmile Matejcic 2 51000 Rijeka Croatia

Correspondence should be addressed to Ana Mestrovic amestrovicunirihr

Received 2 July 2019 Revised 20 December 2019 Accepted 3 February 2020 Published 9 March 2020

Academic Editor Roberto Natella

Copyright copy 2020ETHurCica Vukic et alis is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Discovering the most suitable network structure of the learning domain represents one of the main challenges of knowledgedelivery and acquisition We propose a multidimensional knowledge network (MKN) consisting of three components multilayernetwork and its two projections Each network layer constitutes factual conceptual procedural or metacognitive knowledgewithin the domain of databases as a standard course of computer science study In the MKN layer nodes are concepts orknowledge units and the edges are weighted with regard to Bloomrsquos cognitive learning level e projected network layers arecontrasted with a monolayer network by comparing characterizations of the centrality measures degree centrality closenesscentrality betweenness centrality and eccentricity e study revealed indications of how concepts supported with the highernumber of previously introduced concepts have a dominant role in knowledge acquisition from a view of knowledge structureand content e analysis of communities assortativity coefficient and overlap between MKN layers contributes to betterstructuring of knowledge MKN enables systematic insights into the efficiency of knowledge integration across metacognitivelayers as well as the detection of crucial cognitive concepts that reduceincrease the cognitive load during learning

1 Introduction

e organization of knowledge impacts outcomes inteaching and education where much effort has been investedin the development of instructional strategies that may assiststudents in creating and organizing knowledge structures[1] e representation of knowledge affects the learningprocesses and motivates the design of the teaching materialsespecially in the e-learning systems [1ndash4] Knowledge can berepresented in the explicit form of a map (diagram) latticeor network where conceptual elements are interconnectedforming a cohesive and contingent system [5 6] en theprocess of learning involves understanding what the keyconcepts are and how are they related connected and in-tegrated into a more extensive system Such interconnec-tions between concepts have an essential role in establishingtheir meaning and affecting how concepts are introduced inteaching and how they are acquired in formal teaching and

learning [2] Cognitive load theory suggests that learners canabsorb and retain information effectively only if it is pro-vided in such a way that it does not ldquooverloadrdquo their mentalcapacity [7] Siew et al in [8] review means of networkscience to provide a consistent and powerful approach torepresent the structure and dynamics in cognitive systemsspecifically in semantic memory and mental lexicon hencerevealing internal personrsquos (or studentrsquos) representation Inthis work we investigate the possibilities of external rep-resentation of the knowledge in a selected domain and henceexternal representation in a system ie e-learning system[9]

In educational settings the use of graphical knowledgerepresentation tools like concept maps has been a primarysupporting technique for the formation of organizedknowledge models Consequently concept maps have beenapplied at all levels of teaching as well as in textbooks [1]Since the amount of knowledge increases at an

HindawiComplexityVolume 2020 Article ID 9407162 17 pageshttpsdoiorg10115520209407162

unprecedented pace mapping activities align well with theprocesses of knowledge acquisition as they focus on inte-grating existing with new concepts [10] us throughanalyzing concept mapping difficulties that the learner en-counters there is an opportunity to learn providing thecohesion and contingency of the relational structure ofknowledge and enhancing long-term learning outcomesretention and transfer Mapping of concepts into aninterconnected structure is exceptionally efficient in com-parison to other representation techniques such as outliningor defining ideas for learning about the relations betweenideas [5] Concept maps as knowledge integration tools elicitideas as nodes (concepts) [11] erefore analyses of theglobal structure of the interlinked key concepts can beconducted as the analysis of a complex network

Research in [6] presented the idea of applying complexnetwork theory and measures for the characterization of theconcept maps in order to identify key concepts that providethe cohesion and contingency of the whole network ofconcepts Motivated by research challenges addressed in[6 8] in this study we introduce the multidimensionalknowledge network (MKN) based on the learning outcomesof crucial concepts in the domain While the study ofrepresenting the structure in cognitive systems is mainlytasked with efficient internal representations of the mentalmodel [8 12ndash14] here we aim into the modeling of externalknowledge which can be incorporated into an e-learningsystem of the domain or can be used to assess the studentrsquosprogress in acquiring the subject which is grounded inBloomrsquos taxonomy [7 15 16]

e learning outcomes stem from different knowledgelevels from Revised Bloomrsquos taxonomy [16ndash18] RevisedBloomrsquos taxonomy differentiates factual conceptual pro-cedural and metacognitive knowledge dimensions Factualknowledge captures discrete isolated content elements(terminology and knowledge of specific details and ele-ments) Conceptual knowledge comprises classifications andcategories principles and generalizations and theoriesmodels and structures of concepts Procedural knowledgeincludes skills and algorithms techniques and methods aswell as knowledge of the criteria used to determine andjustify ldquo when to do whatrdquo within specific domains anddisciplines Metacognitive knowledge encompasses strategicknowledge knowledge about cognitive tasks includingcontextual and conditional knowledge and self-knowledge[7]

In this work we study the principles of representing andanalyzing how domain knowledge (concepts) can bemodeled across four knowledge levels in a complex networkframework aiming at the facilitation of knowledge modelingacquisition and transfer performance

We propose a multidimensional knowledge network(MKN) which is based on the multilayer network Multi-dimensional knowledge network is composed of three parts(i) directed and weighted multilayer network (ii) interlayerprojection of multilayer network and (iii) monolayer pro-jection of multilayer network e interlayer projection isconstructed by projecting the edges between layers to thehigher layer in the hierarchy while the monolayer projection

is constructed by projecting all nodes and edges onto onelayeris way the defined model enables systematic insightsinto the knowledge dimensions and efficiency of knowledgeintegration across metacognitive layers as well as the de-tection of key cognitive concepts that reduceincrease thecognitive load in processing information e results ob-tained from MKN analysis can shed light on the causes ofincreased cognitive demands indicate vulnerabilities in theknowledge (more specific and identify knowledge units thatrequire modification of instructional strategies) and con-sequently guide the design and optimization of learningoutcomes For the experimental purposes we evaluateproposedMKN in the field of computer sciencemdashspecificallyfor the databases including standard topics in databasedesign and implementation For the domain of ldquodatabaserdquowe construct the multidimensional knowledge networkfollowing the set of learning outcomes for the databasecourse and provide a detailed analysis of key concepts in theproposed MKN model

In the second section we describe the theoreticalbackground of network science and cognitive complexnetworks Moreover we give an overview of network theoryapplications in cognitive science and knowledge represen-tation and their contributionse third section provides themethodology for multilayer network analysis with the def-inition of qualitative and quantitative measures e fourthsection covers the gathering of experimental data and net-work construction principles for database-domain learningoutcomes e fifth section is dedicated to the reporting anddiscussion of the MKN model e paper concludes in thesixth section with future research plans

2 Background

e emergence of a new paradigm-complexity lies at the coreof the information age reflecting everything that has in-trinsically complex behavior and cannot be described in acomprehensive manner [19] e ability to reason andcomprehend such properties finds its roots in graph theoryand sociology when Paul Erdős and Alfred Renyi researchstudy draw a new multidisciplinary interest into the study ofcomplex networks [19ndash21] e rise of interest in under-standing general properties of complex systems had animpact on a substantial upsurge in the study of inter-connected structures in many disciplines and generally theevolution of network science [8 21ndash33] e analysis of theimmense amount of data due to its exponential growthresulted in novel analytic methods for complex networkanalysis considering global and local network structures aswell as the time-varying and multilayer nature Multilayernetworks explicitly incorporate multiple channels of con-nectivity and constitute the natural environment to describesystems interconnected through different categories ofconnections a layer represents each channel (relationshipactivity and category) and the same node or entity may havedifferent kinds of interactions (different set of neighbors ineach layer) [31 34 35] Our research in the direction ofmultiple layersrsquo analysis is motivated by the fact that mul-tilayered network structure is fundamentally more

2 Complexity

expressive than individual layers [36] ey use multiplexnetworks for the task of predicting the ordering with whichwords are acquired However the similar argumentationand motivation hold in the case of the learning process aswell Furthermore multilayer analysis allows quantificationof distinct phases of learning and multilayered networks

21 Related Work on Conceptual and Cognitive ComplexNetworks Individuals differ in their ability to learn fromexperience to adapt to new situations and overcome chal-lenges to understand simple or complex ideas to solve real-world or abstract problems and to engage in different formsof reasoning and thinking [5 8 17 37 38] Knowledgeacquisition and integration activities are designed to helplearners construct a more coherent understanding by de-veloping criteria for the ideas that they encounter Conceptmaps as knowledge integration tools elicit knowledge ele-ments as nodes (concepts) [5 11] and relations betweenthem as edges us the concept maps are network rep-resentations of the organization of the concepts e ad-vantage of such representations emphasizes the relationalstructure of knowledge where the concepts and principlesare interconnected and where the principles of making theconnections can be explicated [39] Concept mapping isexceptionally efficient in comparison to other techniquessuch as outlining or defining knowledge elements [5]

Schwendimann in [11] indicates that concept maps cansupport knowledge integration processes by eliciting coreideas and connections and making possible clusters or hi-erarchies visible Similarity the authors in [5] report thatgraphic organizers such as concept maps can foster theintegration of fragmented ideas toward an organized inter-connected network of ideas Moreover recent research studyfocusing on the structure of the concept map suggests thatbetter understanding and the high quality of studentsrsquoknowledge are reflected in interconnected and web-likestructures [40] e authors in [41] highlighted the role ofcognition and connection to language complex network bythe principle that human performance can be related tonetwork properties further suggesting that network prop-erties might provide evidence of or have an influence onhuman cognition as related to the acquisition of languageGurevych in [42] proposed a method to generate artificialdefinitions of concepts from a conceptual network of wordsfinding that semantic relatedness of words compensates thelack of definitions in a conceptual hierarchy by generating atextual definition of the concept automatically from aknowledge base e semantic relatedness metric generatedglosses that correlate very well with human judgments ofsemantic relatedness In [39] the authors state that inpractice successful instruction and learning which set itsgoals on the holistic understanding require a deeper un-derstanding of the knowledge as a coherent and connectedstructure Consequently the coherence and connectivity ofsuch a knowledge representation are known to be closelyrelated to knowledge production acquisition and processing

Knowledge processing and acquisition have been de-scribed in the framework of complex networks theory

[22 27 43] e complex network methodology is wellsuited for a description of relations in the conceptualknowledge and dynamics of the retrieval process ofknowledge [8] Network motivated approaches are welladapted to the related problems of knowledge modelingretrieval and acquisition Reported research study ingeneral is indicating the importance of establishing con-nections between words regarding their meaning semanticrelations phonological similarity or syntax [2 8 13 27 44]Hence a lot of knowledge-related studies are intertwinedwith studies of language complex networks [12 35 45ndash48]For instance the authors in [49] consider the lexicalstructure of topics in a course as a monolayer lexical networkof terms e study reveals that extension to deeper con-textual levels by the inclusion of more remote connectionsbetween the terms (although in the same layer) facilitatesthe representation of knowledge and concludes thatmethods are sensitive enough to lexical or semantic featuresof the text

Traditionally researchers [1ndash4 6 34 35 39 42 49ndash53]analyze isolated aspects of the network structures such as thenumber of links to concepts (degree) [51] the number ofcomponents (unconnected parts of the content structure)the subgraph measures and communicability betweennesscentrality measures to derive rankings of different nodesbased on how important each node is in providing cohesionand contingency [6] A study in [51] is based on degreeclustering transit efficiency betweenness and closenesscentrality measures for analysis of the content structure ofvideo lessons and identification of the key aspects of contentstructure concerning student learning gains Siew in [50]analyzed monolayer conceptual networks from the conceptmaps generated by students of psychology and confirmedthat concept networks differed across students and predictedlearning outcomes In short the study concluded thatconcept networks with larger average shortest path lengthswere associated with higher scores hence suggesting thatnetwork science can be used to quantify the conceptualstructure of a learnerrsquos knowledge e common short-coming of listed work is the granularity of the examinedinformation since only single and isolated aspects of theknowledge are investigated in a monolayer setup

Cognitive complex networks establish the foundation forunderstanding the principles for the study of conceptualnetworks in a more comprehensive manner e aim forunderstanding the cognitive processes behind knowledgeconstruction and its acquisition leads to the development ofintegral computational models for cognitive processes inlearning [2] Hence they reach beyond monolayer networksinto multilayered or multiplex structures Research oncognitive networks utilizes the framework of multiplexlexical networks for investigating lexical retrieval frommemory [13] It uses the multiplex network to study how thelayout of word-word similarities in the mental lexicon canlead to priming effects on multiple combined semantic andphonological levels Multiplex lexical networks have provenfundamentally more powerful in investigating the process ofearly word acquisition [13 36 46] and for detection of thecore structure of mental lexicon indicating the significance

Complexity 3

of integrating the importance of multiple word-word rela-tions [14] e conceptual network of the English languagein [54] has also been examined through the means ofcognitive science wherefrom the standpoint of retrieval ofinformation from associative memory the small-worldproperty of the network represents a maximization of re-trieval efficiency

In this work we are aiming to fill the gap of the structuralrepresentation of the knowledge organized according torevised Bloomrsquos taxonomy into factual conceptual proce-dural and metacognitive knowledge employing layers in amultilayer complex network

3 Methodology for MultilayerNetwork Analysis

e study of complex systems has impelled researchers tomove from simple graph representations to more abstractanalyses by including multiple subsystems and layers ofconnectivity [42] Different notions of multilayer networkscan be obtained with regard to various constraints whichgive rise to networks of networks [26] multidimensionalnetworks [28 55] multilayer networks [29 32] multiplexnetworks [13 24 36 45 46 56 57] interacting networks[25] interdependent networks [58] and many others thathave been introduced [34] A theoretical framework ofmultilayer network structures from the literature addresses ageneral form of multilayer network [31 34]

31 Multidimensional Knowledge Network Model In thisresearch we propose a novel integrative model forknowledge representation that enables a multidimensionalanalysis e proposed model is based on the multilayernetwork with two extensions e first extension is definedas the interlayer projection of the initial multilayer networkwhich is constructed by projecting interlayer edges onto onelayer according to a predefined rule e second extension isdefined as the monolayer projection of the initial multilayernetwork which is constructed in a way that all nodes andedges are projected onto one single layer

In this section we give definitions of all these formalmodels and in the next section we provide a context andinterpretation of defined models

According to [34] a multilayer network is defined as apair

M (G C) (1)

where

G Ga α isin 1 M 1113864 1113865 (2)

is a family of networks (graphs) Ga (Vα Eα) called net-work layers of M and

C Eαβ subeV

αtimes V

β α β isin 1 M αne β1113966 1113967 (3)

is the set of interconnections between nodes of different layersGα and Gβ where αne β

Layers are annotated as numbers from the set 1 M whereM is the number of layers e network multilayeredas well can be directed or undirected and weighted orunweighted however the selected network model has to beconsistent for all layers in the multilayer network [35]erefore the whole multilayer network can be defined asdirected or undirected and weighted or unweighted In thisresearch we construct a weighted and directed multilayernetwork

e set of nodes of the network layer Vα is denoted byVα xα

1 xα2 xα

Nα1113966 1113967 and the adjacency matrix of each

layer α is denoted by A[α] (aαij) where

aαij

1 if xαi xα

j1113872 1113873 isin Eα

0 otherwise

⎧⎨

⎩ (4)

for 1le i jleNα and 1le αleM e interlayer adjacencymatrix corresponding to Eαβ is the matrix A[αβ] (ααβij )

defined by

aαβij

1 if xβi x

βj1113872 1113873 isin Eαβ

0 otherwise

⎧⎨

⎩ (5)

Note that in the case of weightedmultilayer network theadjacencymatrices contain corresponding weights instead of1 denoted as A[α] (wα

ij) in the case of intralayer con-nections and A[αβ] (aw

αβij ) in the case of interlayer con-

nections Additionally we define and consider a special typeof directed multilayer network which can be derived fromthe initial multilayer network by projecting interlayer edgesonto one single layer according to the target node

For a given directed multilayer networkM we define aninterlayer projection of M denoted as

ip(M) Gαip α isin 1 M 1113966 1113967 (6)

in a way that for every edge from the set of interlayerconnections e

αβk (xα

i xβj ) isin Eαβ and we shift the starting

node xαi to the layer β hence it becomes the node x

βi Ac-

cordingly the existing edge eαβk is projected to the β layer in

the way that it becomes a new edge eβk (x

βi x

βj ) isin Eα

e result of the projection is a family of networksGα

ip α isin 1 M 1113966 1113967 without interlayer edges e projec-tion rule assures that all interlayer edges are projected ontothe target layer including the projection of a starting node xα

i

to xβi In the continuation of the paper we will refer to the

interlayer projection of M as projection of M or multilayerprojection (MKN projection) in short

Next we define the monolayer projection of the initialmultilayer network M denoted as mp(M) in the way thatwe project all nodes and links from allM layers to one singlelayer In the continuation of the paper we refer to thatnetwork as a monolayer (projection) network in short InSupplementary Material we list the definition of all networkmeasures used for the quantification of a multilayer networka multilayer projection network and a monolayer network

Finally the multidimensional knowledge network isdefined as a triple

MKN (M ip(M) mp(m)) (7)

4 Complexity

32 Modeling the Network of Concepts Given the fact thatthe network of concepts in itsrsquo simplest definition repre-sents a system of connected parts we assume that theknowledge system of any domain (subject) can be modeledas a complex network of knowledge In the context of theselected domain (course) the concepts are nodes and theirrelationships may be represented as edges that connectknowledge units e edges are directed reflecting the de-pendence of a hierarchical ordering that follows from thenavigational path of learning in which one unit of knowledgeis introduced (acquired) before the other Course designreflects an effective instructional plan aiming to optimizealignment between learning objectives assessments andinstructional activities organized in a specific ordering oftopics (ie crucial concepts of the domain) Hence theassumption is that the network of concepts follows theordering of efficient knowledge acquisition through thelearning process [1ndash4 6 39 49]

Determining learning outcomes start from a higher leveland at first determines the learning outcomes for the studyprogram then for the module and then for the group ofsubjects followed by outcomes of individual subjects whichare finally decomposed into knowledge units [17 59] eplanned outcomes (learning objectives) can be expressed interms of the content (expressed as nouns) and the cognitiveprocess needed (expressed as verbs) In this sense [7] theoriginal Bloomrsquos taxonomy was one-dimensional becausethe categories contained only nouns (objects) and verbs(actions) e verb generally refers to the actions associatedwith the intended cognitive process e object generallydescribes the knowledge students are expected to acquire orconstruct e cognitive process dimension represents acontinuum of increasing cognitive complexitymdashfrom re-member to create [16] Each node is a concept or knowledgeunit and it is a semantic part of a learning objective erelation between two nodes (two knowledge units) isestablished if these two knowledge units appear in the same(common) learning objective (outcome) and vice versa thelearning objective is a relation between two concepts (nodes)concerning the cognitive process and the knowledge di-mensions according to revised Bloomrsquos taxonomy RevisedBloomrsquos taxonomy differentiates between four dimensionsof knowledge factual conceptual procedural or meta-cognitive knowledge [16] According to this taxonomy eachlevel of knowledge corresponds to the level of cognitiveprocess (load) so a student can remember factual or pro-cedural knowledge understand conceptual or metacognitiveknowledge or analyze metacognitive or factual knowledge[15ndash18] Moreover according to the revised version ofBloomrsquos taxonomy there are six dimensions of the cognitiveprocess remembering understanding applying analyzingevaluating and creating

Hence revised Bloomrsquos taxonomy can be represented asa multilayer network where each layer models one di-mension of knowledge To this end we propose a multi-dimensional knowledge network (MKN) formally definedin the previous section

First we define a multilayer network M which consistsof four layers Each layer represents one dimension (the

knowledge level) of Bloomrsquos taxonomy factual conceptualprocedural and metacognitive knowledge In each layernodes represent knowledge units defined according to thelearning outcome It is important to note that one node maybelong to different layers since it can be associated withdifferent learning outcomesmdashhence different levels ofBloomrsquos taxonomy Still it is not required that all nodes(knowledge units) are present on all layers ie someconcepts are for instance present at factual and conceptuallayers and not at procedural and metacognitive oneserefore we opt for modeling of the multilayer and not forthe multiplex network

Nodes are connected between each other in one layer(intralayer edges) and across layers (interlayer edges) Anedge between two nodes (knowledge units) is constructed ifthere is a learning outcome in which the first knowledge unitis connected to the second knowledge unit (in terms ofsequence which follows the order of knowledge acquisitionwithin the domain) e edges are weighted with regard toBloomrsquos cognitive process dimensions and weights of thecorresponding edge depend on the cognitive learning level tobe achieved through the learning process that includes twonodes (knowledge units) Hence remembering receives aweight of 1 understanding 2 applying 3 analyzing 4evaluating 5 and creating 6

Finally we also construct the monolayer projection ofthe multilayer network M Hence a monolayer networkcontains all nodes and edges weighted with respect to thecognitive process dimension and analyzed as a classicalcomplex network providing information about the globalnetwork properties Results of monolayer network analysisrepresent ldquoinitial rawrdquo data that can be used to identify whichtop-ranked (monolayer) nodes according to differences inresults of monolayer and multilayer analysis should bereviewed concerning its knowledge dimension identity inmultilayer and projected components of the MKN Forexample high in-degree nodes in a monolayer networkshould indicate nodes at the factual knowledge dimensionsince it refers to isolated facts and details of concepts at thevery early stage of learning In the continuation themonolayer network is compared to projected layers in orderto quantify the specific properties of each layer is isgrounded in the theory of learning When the units of thetask are being processed simultaneously or when the contenthas a high degree of interactivity among elements the highcognitive load will be imposed on the student even when thenumber of interreacting elements is relatively small As theintrinsic cognitive load is essential for the achievement ofspecific learning goals (understanding of the problemconstruction of higher structures of knowledge and theirflexibility) it must be within the capacity of workingmemory (ie within limits of intrinsic cognitive load)

Figure 1 (adapted from [16]) illustrates an edge con-struction rule for the learning outcome ldquoStudents willdifferentiate between terms database and DBMSrdquo islearning outcome results with two nodes (knowledge units)database and DBMS with a verb differentiate and establishesthe intralayer edge in 4th (analyze) cognitive process di-mension on the factual layer Similarly for the outcome

Complexity 5

ldquoStudent will explain the correlation between 4NF andMultivalued dependencyrdquo We define interlayer edge (1)node-4NF at procedural layer and (2) nodendashmultivalueddependency at conceptual layer with weight 2 (verb un-derstand) It is worth noticing that knowledge units areassumed to lie along a continuum from concrete (factual) toabstract (metacognitive) e conceptual and proceduralcategories overlap in terms of abstractness e verb gen-erally describes the intended cognitive process and theconcept generally describes the knowledge students areexpected to acquire [16]

e process of the construction of the multidimensionalknowledge network MKN with all three componentsM ip(M) andmp(M) and the process of the analysis canbe summarized in the following ten steps Formalization ofthe process in ten steps can serve as a recipe for the transferof the proposed methodology into a new domain

Step 1 Selecting the knowledge domain and defining ahierarchical list of concepts (knowledge units of the in-structional plan for domain) For instance if we decide toconstruct the MKN for the database domain we will defineconcepts like tables relations normal form and index

Step 2 Designing relevant learning outcomes from relationsbetween concepts and aligning them with respect to Bloomrsquostaxonomy of cognition For example from the learningoutcome ldquoStudents will explain the use of database nor-malization as the systematic approach of decomposing tablesrdquowe can derive a relation decomposing (tables and databasenormalization)

Step 3 Constructing the first component of the MKN amultilayer network M again nodes are concepts from the

domain and one node can belong to one or more layersaccording to the learning outcomes that include that concept

Step 4 Constructing the second component of the MKN aninterlayer projection of M where we project the interlayeredges into the target layer as defined in Section 31 As theresult we have ip(M) which consist of four network layerswithout interlayer connections

Step 5 Constructing the third component of the MKN amonolayer projection of M mp(M) where we project allnodes and edges onto one single layer As the result weobtain mp(M) which consists of one network layer with allthe nodes and edges

Step 6 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the global level

Step 7 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the local level in terms of iden-tifying key concepts on each layer

Step 8 Identifying communities aiming for the justificationof knowledge organization and hierarchical partitioning ofinstructional content into coherent groups of concepts

Step 9 Analyzing four layers in terms of assortativitymixing

Step 10 Analyzing four projected layers in terms of nodeand edge overlapping

Theknowledgedimension-LAYER

Factual

Conceptual

Procedural

Metacognitive

Knowledge dimension [LAYER]

The Cognitive process dimension - WEIGHT

(1) Factual(2) Conceptual(3) Procedural(4) Metacognitive

(2)Understand

(1)Remember

(3)Apply

(4)Analyze

(5)Evaluate

(6)Create

Cognitive process dimension [WEIGHT](1) Remember(2) Understand(3) Apply(4) Analyze(5) Evaluate(6) Create

Knowledge unitDatabase DBMS

Verbdifferentitate

Instructional objectiveStudents will differentiate between terms Database and DBMS

xStudent will differentiatebetween term Database

and DBMS

Figure 1 Construction of edges for the outcome ldquoStudents will differentiate between terms database and DBMSrdquo results with two nodes(knowledge units) database and DBMS and the verb differentiate establishing the edge with weight 4 in cognitive process dimension(analyze) at the layer of factual knowledge

6 Complexity

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

expressive than individual layers [36] ey use multiplexnetworks for the task of predicting the ordering with whichwords are acquired However the similar argumentationand motivation hold in the case of the learning process aswell Furthermore multilayer analysis allows quantificationof distinct phases of learning and multilayered networks

21 Related Work on Conceptual and Cognitive ComplexNetworks Individuals differ in their ability to learn fromexperience to adapt to new situations and overcome chal-lenges to understand simple or complex ideas to solve real-world or abstract problems and to engage in different formsof reasoning and thinking [5 8 17 37 38] Knowledgeacquisition and integration activities are designed to helplearners construct a more coherent understanding by de-veloping criteria for the ideas that they encounter Conceptmaps as knowledge integration tools elicit knowledge ele-ments as nodes (concepts) [5 11] and relations betweenthem as edges us the concept maps are network rep-resentations of the organization of the concepts e ad-vantage of such representations emphasizes the relationalstructure of knowledge where the concepts and principlesare interconnected and where the principles of making theconnections can be explicated [39] Concept mapping isexceptionally efficient in comparison to other techniquessuch as outlining or defining knowledge elements [5]

Schwendimann in [11] indicates that concept maps cansupport knowledge integration processes by eliciting coreideas and connections and making possible clusters or hi-erarchies visible Similarity the authors in [5] report thatgraphic organizers such as concept maps can foster theintegration of fragmented ideas toward an organized inter-connected network of ideas Moreover recent research studyfocusing on the structure of the concept map suggests thatbetter understanding and the high quality of studentsrsquoknowledge are reflected in interconnected and web-likestructures [40] e authors in [41] highlighted the role ofcognition and connection to language complex network bythe principle that human performance can be related tonetwork properties further suggesting that network prop-erties might provide evidence of or have an influence onhuman cognition as related to the acquisition of languageGurevych in [42] proposed a method to generate artificialdefinitions of concepts from a conceptual network of wordsfinding that semantic relatedness of words compensates thelack of definitions in a conceptual hierarchy by generating atextual definition of the concept automatically from aknowledge base e semantic relatedness metric generatedglosses that correlate very well with human judgments ofsemantic relatedness In [39] the authors state that inpractice successful instruction and learning which set itsgoals on the holistic understanding require a deeper un-derstanding of the knowledge as a coherent and connectedstructure Consequently the coherence and connectivity ofsuch a knowledge representation are known to be closelyrelated to knowledge production acquisition and processing

Knowledge processing and acquisition have been de-scribed in the framework of complex networks theory

[22 27 43] e complex network methodology is wellsuited for a description of relations in the conceptualknowledge and dynamics of the retrieval process ofknowledge [8] Network motivated approaches are welladapted to the related problems of knowledge modelingretrieval and acquisition Reported research study ingeneral is indicating the importance of establishing con-nections between words regarding their meaning semanticrelations phonological similarity or syntax [2 8 13 27 44]Hence a lot of knowledge-related studies are intertwinedwith studies of language complex networks [12 35 45ndash48]For instance the authors in [49] consider the lexicalstructure of topics in a course as a monolayer lexical networkof terms e study reveals that extension to deeper con-textual levels by the inclusion of more remote connectionsbetween the terms (although in the same layer) facilitatesthe representation of knowledge and concludes thatmethods are sensitive enough to lexical or semantic featuresof the text

Traditionally researchers [1ndash4 6 34 35 39 42 49ndash53]analyze isolated aspects of the network structures such as thenumber of links to concepts (degree) [51] the number ofcomponents (unconnected parts of the content structure)the subgraph measures and communicability betweennesscentrality measures to derive rankings of different nodesbased on how important each node is in providing cohesionand contingency [6] A study in [51] is based on degreeclustering transit efficiency betweenness and closenesscentrality measures for analysis of the content structure ofvideo lessons and identification of the key aspects of contentstructure concerning student learning gains Siew in [50]analyzed monolayer conceptual networks from the conceptmaps generated by students of psychology and confirmedthat concept networks differed across students and predictedlearning outcomes In short the study concluded thatconcept networks with larger average shortest path lengthswere associated with higher scores hence suggesting thatnetwork science can be used to quantify the conceptualstructure of a learnerrsquos knowledge e common short-coming of listed work is the granularity of the examinedinformation since only single and isolated aspects of theknowledge are investigated in a monolayer setup

Cognitive complex networks establish the foundation forunderstanding the principles for the study of conceptualnetworks in a more comprehensive manner e aim forunderstanding the cognitive processes behind knowledgeconstruction and its acquisition leads to the development ofintegral computational models for cognitive processes inlearning [2] Hence they reach beyond monolayer networksinto multilayered or multiplex structures Research oncognitive networks utilizes the framework of multiplexlexical networks for investigating lexical retrieval frommemory [13] It uses the multiplex network to study how thelayout of word-word similarities in the mental lexicon canlead to priming effects on multiple combined semantic andphonological levels Multiplex lexical networks have provenfundamentally more powerful in investigating the process ofearly word acquisition [13 36 46] and for detection of thecore structure of mental lexicon indicating the significance

Complexity 3

of integrating the importance of multiple word-word rela-tions [14] e conceptual network of the English languagein [54] has also been examined through the means ofcognitive science wherefrom the standpoint of retrieval ofinformation from associative memory the small-worldproperty of the network represents a maximization of re-trieval efficiency

In this work we are aiming to fill the gap of the structuralrepresentation of the knowledge organized according torevised Bloomrsquos taxonomy into factual conceptual proce-dural and metacognitive knowledge employing layers in amultilayer complex network

3 Methodology for MultilayerNetwork Analysis

e study of complex systems has impelled researchers tomove from simple graph representations to more abstractanalyses by including multiple subsystems and layers ofconnectivity [42] Different notions of multilayer networkscan be obtained with regard to various constraints whichgive rise to networks of networks [26] multidimensionalnetworks [28 55] multilayer networks [29 32] multiplexnetworks [13 24 36 45 46 56 57] interacting networks[25] interdependent networks [58] and many others thathave been introduced [34] A theoretical framework ofmultilayer network structures from the literature addresses ageneral form of multilayer network [31 34]

31 Multidimensional Knowledge Network Model In thisresearch we propose a novel integrative model forknowledge representation that enables a multidimensionalanalysis e proposed model is based on the multilayernetwork with two extensions e first extension is definedas the interlayer projection of the initial multilayer networkwhich is constructed by projecting interlayer edges onto onelayer according to a predefined rule e second extension isdefined as the monolayer projection of the initial multilayernetwork which is constructed in a way that all nodes andedges are projected onto one single layer

In this section we give definitions of all these formalmodels and in the next section we provide a context andinterpretation of defined models

According to [34] a multilayer network is defined as apair

M (G C) (1)

where

G Ga α isin 1 M 1113864 1113865 (2)

is a family of networks (graphs) Ga (Vα Eα) called net-work layers of M and

C Eαβ subeV

αtimes V

β α β isin 1 M αne β1113966 1113967 (3)

is the set of interconnections between nodes of different layersGα and Gβ where αne β

Layers are annotated as numbers from the set 1 M whereM is the number of layers e network multilayeredas well can be directed or undirected and weighted orunweighted however the selected network model has to beconsistent for all layers in the multilayer network [35]erefore the whole multilayer network can be defined asdirected or undirected and weighted or unweighted In thisresearch we construct a weighted and directed multilayernetwork

e set of nodes of the network layer Vα is denoted byVα xα

1 xα2 xα

Nα1113966 1113967 and the adjacency matrix of each

layer α is denoted by A[α] (aαij) where

aαij

1 if xαi xα

j1113872 1113873 isin Eα

0 otherwise

⎧⎨

⎩ (4)

for 1le i jleNα and 1le αleM e interlayer adjacencymatrix corresponding to Eαβ is the matrix A[αβ] (ααβij )

defined by

aαβij

1 if xβi x

βj1113872 1113873 isin Eαβ

0 otherwise

⎧⎨

⎩ (5)

Note that in the case of weightedmultilayer network theadjacencymatrices contain corresponding weights instead of1 denoted as A[α] (wα

ij) in the case of intralayer con-nections and A[αβ] (aw

αβij ) in the case of interlayer con-

nections Additionally we define and consider a special typeof directed multilayer network which can be derived fromthe initial multilayer network by projecting interlayer edgesonto one single layer according to the target node

For a given directed multilayer networkM we define aninterlayer projection of M denoted as

ip(M) Gαip α isin 1 M 1113966 1113967 (6)

in a way that for every edge from the set of interlayerconnections e

αβk (xα

i xβj ) isin Eαβ and we shift the starting

node xαi to the layer β hence it becomes the node x

βi Ac-

cordingly the existing edge eαβk is projected to the β layer in

the way that it becomes a new edge eβk (x

βi x

βj ) isin Eα

e result of the projection is a family of networksGα

ip α isin 1 M 1113966 1113967 without interlayer edges e projec-tion rule assures that all interlayer edges are projected ontothe target layer including the projection of a starting node xα

i

to xβi In the continuation of the paper we will refer to the

interlayer projection of M as projection of M or multilayerprojection (MKN projection) in short

Next we define the monolayer projection of the initialmultilayer network M denoted as mp(M) in the way thatwe project all nodes and links from allM layers to one singlelayer In the continuation of the paper we refer to thatnetwork as a monolayer (projection) network in short InSupplementary Material we list the definition of all networkmeasures used for the quantification of a multilayer networka multilayer projection network and a monolayer network

Finally the multidimensional knowledge network isdefined as a triple

MKN (M ip(M) mp(m)) (7)

4 Complexity

32 Modeling the Network of Concepts Given the fact thatthe network of concepts in itsrsquo simplest definition repre-sents a system of connected parts we assume that theknowledge system of any domain (subject) can be modeledas a complex network of knowledge In the context of theselected domain (course) the concepts are nodes and theirrelationships may be represented as edges that connectknowledge units e edges are directed reflecting the de-pendence of a hierarchical ordering that follows from thenavigational path of learning in which one unit of knowledgeis introduced (acquired) before the other Course designreflects an effective instructional plan aiming to optimizealignment between learning objectives assessments andinstructional activities organized in a specific ordering oftopics (ie crucial concepts of the domain) Hence theassumption is that the network of concepts follows theordering of efficient knowledge acquisition through thelearning process [1ndash4 6 39 49]

Determining learning outcomes start from a higher leveland at first determines the learning outcomes for the studyprogram then for the module and then for the group ofsubjects followed by outcomes of individual subjects whichare finally decomposed into knowledge units [17 59] eplanned outcomes (learning objectives) can be expressed interms of the content (expressed as nouns) and the cognitiveprocess needed (expressed as verbs) In this sense [7] theoriginal Bloomrsquos taxonomy was one-dimensional becausethe categories contained only nouns (objects) and verbs(actions) e verb generally refers to the actions associatedwith the intended cognitive process e object generallydescribes the knowledge students are expected to acquire orconstruct e cognitive process dimension represents acontinuum of increasing cognitive complexitymdashfrom re-member to create [16] Each node is a concept or knowledgeunit and it is a semantic part of a learning objective erelation between two nodes (two knowledge units) isestablished if these two knowledge units appear in the same(common) learning objective (outcome) and vice versa thelearning objective is a relation between two concepts (nodes)concerning the cognitive process and the knowledge di-mensions according to revised Bloomrsquos taxonomy RevisedBloomrsquos taxonomy differentiates between four dimensionsof knowledge factual conceptual procedural or meta-cognitive knowledge [16] According to this taxonomy eachlevel of knowledge corresponds to the level of cognitiveprocess (load) so a student can remember factual or pro-cedural knowledge understand conceptual or metacognitiveknowledge or analyze metacognitive or factual knowledge[15ndash18] Moreover according to the revised version ofBloomrsquos taxonomy there are six dimensions of the cognitiveprocess remembering understanding applying analyzingevaluating and creating

Hence revised Bloomrsquos taxonomy can be represented asa multilayer network where each layer models one di-mension of knowledge To this end we propose a multi-dimensional knowledge network (MKN) formally definedin the previous section

First we define a multilayer network M which consistsof four layers Each layer represents one dimension (the

knowledge level) of Bloomrsquos taxonomy factual conceptualprocedural and metacognitive knowledge In each layernodes represent knowledge units defined according to thelearning outcome It is important to note that one node maybelong to different layers since it can be associated withdifferent learning outcomesmdashhence different levels ofBloomrsquos taxonomy Still it is not required that all nodes(knowledge units) are present on all layers ie someconcepts are for instance present at factual and conceptuallayers and not at procedural and metacognitive oneserefore we opt for modeling of the multilayer and not forthe multiplex network

Nodes are connected between each other in one layer(intralayer edges) and across layers (interlayer edges) Anedge between two nodes (knowledge units) is constructed ifthere is a learning outcome in which the first knowledge unitis connected to the second knowledge unit (in terms ofsequence which follows the order of knowledge acquisitionwithin the domain) e edges are weighted with regard toBloomrsquos cognitive process dimensions and weights of thecorresponding edge depend on the cognitive learning level tobe achieved through the learning process that includes twonodes (knowledge units) Hence remembering receives aweight of 1 understanding 2 applying 3 analyzing 4evaluating 5 and creating 6

Finally we also construct the monolayer projection ofthe multilayer network M Hence a monolayer networkcontains all nodes and edges weighted with respect to thecognitive process dimension and analyzed as a classicalcomplex network providing information about the globalnetwork properties Results of monolayer network analysisrepresent ldquoinitial rawrdquo data that can be used to identify whichtop-ranked (monolayer) nodes according to differences inresults of monolayer and multilayer analysis should bereviewed concerning its knowledge dimension identity inmultilayer and projected components of the MKN Forexample high in-degree nodes in a monolayer networkshould indicate nodes at the factual knowledge dimensionsince it refers to isolated facts and details of concepts at thevery early stage of learning In the continuation themonolayer network is compared to projected layers in orderto quantify the specific properties of each layer is isgrounded in the theory of learning When the units of thetask are being processed simultaneously or when the contenthas a high degree of interactivity among elements the highcognitive load will be imposed on the student even when thenumber of interreacting elements is relatively small As theintrinsic cognitive load is essential for the achievement ofspecific learning goals (understanding of the problemconstruction of higher structures of knowledge and theirflexibility) it must be within the capacity of workingmemory (ie within limits of intrinsic cognitive load)

Figure 1 (adapted from [16]) illustrates an edge con-struction rule for the learning outcome ldquoStudents willdifferentiate between terms database and DBMSrdquo islearning outcome results with two nodes (knowledge units)database and DBMS with a verb differentiate and establishesthe intralayer edge in 4th (analyze) cognitive process di-mension on the factual layer Similarly for the outcome

Complexity 5

ldquoStudent will explain the correlation between 4NF andMultivalued dependencyrdquo We define interlayer edge (1)node-4NF at procedural layer and (2) nodendashmultivalueddependency at conceptual layer with weight 2 (verb un-derstand) It is worth noticing that knowledge units areassumed to lie along a continuum from concrete (factual) toabstract (metacognitive) e conceptual and proceduralcategories overlap in terms of abstractness e verb gen-erally describes the intended cognitive process and theconcept generally describes the knowledge students areexpected to acquire [16]

e process of the construction of the multidimensionalknowledge network MKN with all three componentsM ip(M) andmp(M) and the process of the analysis canbe summarized in the following ten steps Formalization ofthe process in ten steps can serve as a recipe for the transferof the proposed methodology into a new domain

Step 1 Selecting the knowledge domain and defining ahierarchical list of concepts (knowledge units of the in-structional plan for domain) For instance if we decide toconstruct the MKN for the database domain we will defineconcepts like tables relations normal form and index

Step 2 Designing relevant learning outcomes from relationsbetween concepts and aligning them with respect to Bloomrsquostaxonomy of cognition For example from the learningoutcome ldquoStudents will explain the use of database nor-malization as the systematic approach of decomposing tablesrdquowe can derive a relation decomposing (tables and databasenormalization)

Step 3 Constructing the first component of the MKN amultilayer network M again nodes are concepts from the

domain and one node can belong to one or more layersaccording to the learning outcomes that include that concept

Step 4 Constructing the second component of the MKN aninterlayer projection of M where we project the interlayeredges into the target layer as defined in Section 31 As theresult we have ip(M) which consist of four network layerswithout interlayer connections

Step 5 Constructing the third component of the MKN amonolayer projection of M mp(M) where we project allnodes and edges onto one single layer As the result weobtain mp(M) which consists of one network layer with allthe nodes and edges

Step 6 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the global level

Step 7 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the local level in terms of iden-tifying key concepts on each layer

Step 8 Identifying communities aiming for the justificationof knowledge organization and hierarchical partitioning ofinstructional content into coherent groups of concepts

Step 9 Analyzing four layers in terms of assortativitymixing

Step 10 Analyzing four projected layers in terms of nodeand edge overlapping

Theknowledgedimension-LAYER

Factual

Conceptual

Procedural

Metacognitive

Knowledge dimension [LAYER]

The Cognitive process dimension - WEIGHT

(1) Factual(2) Conceptual(3) Procedural(4) Metacognitive

(2)Understand

(1)Remember

(3)Apply

(4)Analyze

(5)Evaluate

(6)Create

Cognitive process dimension [WEIGHT](1) Remember(2) Understand(3) Apply(4) Analyze(5) Evaluate(6) Create

Knowledge unitDatabase DBMS

Verbdifferentitate

Instructional objectiveStudents will differentiate between terms Database and DBMS

xStudent will differentiatebetween term Database

and DBMS

Figure 1 Construction of edges for the outcome ldquoStudents will differentiate between terms database and DBMSrdquo results with two nodes(knowledge units) database and DBMS and the verb differentiate establishing the edge with weight 4 in cognitive process dimension(analyze) at the layer of factual knowledge

6 Complexity

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

of integrating the importance of multiple word-word rela-tions [14] e conceptual network of the English languagein [54] has also been examined through the means ofcognitive science wherefrom the standpoint of retrieval ofinformation from associative memory the small-worldproperty of the network represents a maximization of re-trieval efficiency

In this work we are aiming to fill the gap of the structuralrepresentation of the knowledge organized according torevised Bloomrsquos taxonomy into factual conceptual proce-dural and metacognitive knowledge employing layers in amultilayer complex network

3 Methodology for MultilayerNetwork Analysis

e study of complex systems has impelled researchers tomove from simple graph representations to more abstractanalyses by including multiple subsystems and layers ofconnectivity [42] Different notions of multilayer networkscan be obtained with regard to various constraints whichgive rise to networks of networks [26] multidimensionalnetworks [28 55] multilayer networks [29 32] multiplexnetworks [13 24 36 45 46 56 57] interacting networks[25] interdependent networks [58] and many others thathave been introduced [34] A theoretical framework ofmultilayer network structures from the literature addresses ageneral form of multilayer network [31 34]

31 Multidimensional Knowledge Network Model In thisresearch we propose a novel integrative model forknowledge representation that enables a multidimensionalanalysis e proposed model is based on the multilayernetwork with two extensions e first extension is definedas the interlayer projection of the initial multilayer networkwhich is constructed by projecting interlayer edges onto onelayer according to a predefined rule e second extension isdefined as the monolayer projection of the initial multilayernetwork which is constructed in a way that all nodes andedges are projected onto one single layer

In this section we give definitions of all these formalmodels and in the next section we provide a context andinterpretation of defined models

According to [34] a multilayer network is defined as apair

M (G C) (1)

where

G Ga α isin 1 M 1113864 1113865 (2)

is a family of networks (graphs) Ga (Vα Eα) called net-work layers of M and

C Eαβ subeV

αtimes V

β α β isin 1 M αne β1113966 1113967 (3)

is the set of interconnections between nodes of different layersGα and Gβ where αne β

Layers are annotated as numbers from the set 1 M whereM is the number of layers e network multilayeredas well can be directed or undirected and weighted orunweighted however the selected network model has to beconsistent for all layers in the multilayer network [35]erefore the whole multilayer network can be defined asdirected or undirected and weighted or unweighted In thisresearch we construct a weighted and directed multilayernetwork

e set of nodes of the network layer Vα is denoted byVα xα

1 xα2 xα

Nα1113966 1113967 and the adjacency matrix of each

layer α is denoted by A[α] (aαij) where

aαij

1 if xαi xα

j1113872 1113873 isin Eα

0 otherwise

⎧⎨

⎩ (4)

for 1le i jleNα and 1le αleM e interlayer adjacencymatrix corresponding to Eαβ is the matrix A[αβ] (ααβij )

defined by

aαβij

1 if xβi x

βj1113872 1113873 isin Eαβ

0 otherwise

⎧⎨

⎩ (5)

Note that in the case of weightedmultilayer network theadjacencymatrices contain corresponding weights instead of1 denoted as A[α] (wα

ij) in the case of intralayer con-nections and A[αβ] (aw

αβij ) in the case of interlayer con-

nections Additionally we define and consider a special typeof directed multilayer network which can be derived fromthe initial multilayer network by projecting interlayer edgesonto one single layer according to the target node

For a given directed multilayer networkM we define aninterlayer projection of M denoted as

ip(M) Gαip α isin 1 M 1113966 1113967 (6)

in a way that for every edge from the set of interlayerconnections e

αβk (xα

i xβj ) isin Eαβ and we shift the starting

node xαi to the layer β hence it becomes the node x

βi Ac-

cordingly the existing edge eαβk is projected to the β layer in

the way that it becomes a new edge eβk (x

βi x

βj ) isin Eα

e result of the projection is a family of networksGα

ip α isin 1 M 1113966 1113967 without interlayer edges e projec-tion rule assures that all interlayer edges are projected ontothe target layer including the projection of a starting node xα

i

to xβi In the continuation of the paper we will refer to the

interlayer projection of M as projection of M or multilayerprojection (MKN projection) in short

Next we define the monolayer projection of the initialmultilayer network M denoted as mp(M) in the way thatwe project all nodes and links from allM layers to one singlelayer In the continuation of the paper we refer to thatnetwork as a monolayer (projection) network in short InSupplementary Material we list the definition of all networkmeasures used for the quantification of a multilayer networka multilayer projection network and a monolayer network

Finally the multidimensional knowledge network isdefined as a triple

MKN (M ip(M) mp(m)) (7)

4 Complexity

32 Modeling the Network of Concepts Given the fact thatthe network of concepts in itsrsquo simplest definition repre-sents a system of connected parts we assume that theknowledge system of any domain (subject) can be modeledas a complex network of knowledge In the context of theselected domain (course) the concepts are nodes and theirrelationships may be represented as edges that connectknowledge units e edges are directed reflecting the de-pendence of a hierarchical ordering that follows from thenavigational path of learning in which one unit of knowledgeis introduced (acquired) before the other Course designreflects an effective instructional plan aiming to optimizealignment between learning objectives assessments andinstructional activities organized in a specific ordering oftopics (ie crucial concepts of the domain) Hence theassumption is that the network of concepts follows theordering of efficient knowledge acquisition through thelearning process [1ndash4 6 39 49]

Determining learning outcomes start from a higher leveland at first determines the learning outcomes for the studyprogram then for the module and then for the group ofsubjects followed by outcomes of individual subjects whichare finally decomposed into knowledge units [17 59] eplanned outcomes (learning objectives) can be expressed interms of the content (expressed as nouns) and the cognitiveprocess needed (expressed as verbs) In this sense [7] theoriginal Bloomrsquos taxonomy was one-dimensional becausethe categories contained only nouns (objects) and verbs(actions) e verb generally refers to the actions associatedwith the intended cognitive process e object generallydescribes the knowledge students are expected to acquire orconstruct e cognitive process dimension represents acontinuum of increasing cognitive complexitymdashfrom re-member to create [16] Each node is a concept or knowledgeunit and it is a semantic part of a learning objective erelation between two nodes (two knowledge units) isestablished if these two knowledge units appear in the same(common) learning objective (outcome) and vice versa thelearning objective is a relation between two concepts (nodes)concerning the cognitive process and the knowledge di-mensions according to revised Bloomrsquos taxonomy RevisedBloomrsquos taxonomy differentiates between four dimensionsof knowledge factual conceptual procedural or meta-cognitive knowledge [16] According to this taxonomy eachlevel of knowledge corresponds to the level of cognitiveprocess (load) so a student can remember factual or pro-cedural knowledge understand conceptual or metacognitiveknowledge or analyze metacognitive or factual knowledge[15ndash18] Moreover according to the revised version ofBloomrsquos taxonomy there are six dimensions of the cognitiveprocess remembering understanding applying analyzingevaluating and creating

Hence revised Bloomrsquos taxonomy can be represented asa multilayer network where each layer models one di-mension of knowledge To this end we propose a multi-dimensional knowledge network (MKN) formally definedin the previous section

First we define a multilayer network M which consistsof four layers Each layer represents one dimension (the

knowledge level) of Bloomrsquos taxonomy factual conceptualprocedural and metacognitive knowledge In each layernodes represent knowledge units defined according to thelearning outcome It is important to note that one node maybelong to different layers since it can be associated withdifferent learning outcomesmdashhence different levels ofBloomrsquos taxonomy Still it is not required that all nodes(knowledge units) are present on all layers ie someconcepts are for instance present at factual and conceptuallayers and not at procedural and metacognitive oneserefore we opt for modeling of the multilayer and not forthe multiplex network

Nodes are connected between each other in one layer(intralayer edges) and across layers (interlayer edges) Anedge between two nodes (knowledge units) is constructed ifthere is a learning outcome in which the first knowledge unitis connected to the second knowledge unit (in terms ofsequence which follows the order of knowledge acquisitionwithin the domain) e edges are weighted with regard toBloomrsquos cognitive process dimensions and weights of thecorresponding edge depend on the cognitive learning level tobe achieved through the learning process that includes twonodes (knowledge units) Hence remembering receives aweight of 1 understanding 2 applying 3 analyzing 4evaluating 5 and creating 6

Finally we also construct the monolayer projection ofthe multilayer network M Hence a monolayer networkcontains all nodes and edges weighted with respect to thecognitive process dimension and analyzed as a classicalcomplex network providing information about the globalnetwork properties Results of monolayer network analysisrepresent ldquoinitial rawrdquo data that can be used to identify whichtop-ranked (monolayer) nodes according to differences inresults of monolayer and multilayer analysis should bereviewed concerning its knowledge dimension identity inmultilayer and projected components of the MKN Forexample high in-degree nodes in a monolayer networkshould indicate nodes at the factual knowledge dimensionsince it refers to isolated facts and details of concepts at thevery early stage of learning In the continuation themonolayer network is compared to projected layers in orderto quantify the specific properties of each layer is isgrounded in the theory of learning When the units of thetask are being processed simultaneously or when the contenthas a high degree of interactivity among elements the highcognitive load will be imposed on the student even when thenumber of interreacting elements is relatively small As theintrinsic cognitive load is essential for the achievement ofspecific learning goals (understanding of the problemconstruction of higher structures of knowledge and theirflexibility) it must be within the capacity of workingmemory (ie within limits of intrinsic cognitive load)

Figure 1 (adapted from [16]) illustrates an edge con-struction rule for the learning outcome ldquoStudents willdifferentiate between terms database and DBMSrdquo islearning outcome results with two nodes (knowledge units)database and DBMS with a verb differentiate and establishesthe intralayer edge in 4th (analyze) cognitive process di-mension on the factual layer Similarly for the outcome

Complexity 5

ldquoStudent will explain the correlation between 4NF andMultivalued dependencyrdquo We define interlayer edge (1)node-4NF at procedural layer and (2) nodendashmultivalueddependency at conceptual layer with weight 2 (verb un-derstand) It is worth noticing that knowledge units areassumed to lie along a continuum from concrete (factual) toabstract (metacognitive) e conceptual and proceduralcategories overlap in terms of abstractness e verb gen-erally describes the intended cognitive process and theconcept generally describes the knowledge students areexpected to acquire [16]

e process of the construction of the multidimensionalknowledge network MKN with all three componentsM ip(M) andmp(M) and the process of the analysis canbe summarized in the following ten steps Formalization ofthe process in ten steps can serve as a recipe for the transferof the proposed methodology into a new domain

Step 1 Selecting the knowledge domain and defining ahierarchical list of concepts (knowledge units of the in-structional plan for domain) For instance if we decide toconstruct the MKN for the database domain we will defineconcepts like tables relations normal form and index

Step 2 Designing relevant learning outcomes from relationsbetween concepts and aligning them with respect to Bloomrsquostaxonomy of cognition For example from the learningoutcome ldquoStudents will explain the use of database nor-malization as the systematic approach of decomposing tablesrdquowe can derive a relation decomposing (tables and databasenormalization)

Step 3 Constructing the first component of the MKN amultilayer network M again nodes are concepts from the

domain and one node can belong to one or more layersaccording to the learning outcomes that include that concept

Step 4 Constructing the second component of the MKN aninterlayer projection of M where we project the interlayeredges into the target layer as defined in Section 31 As theresult we have ip(M) which consist of four network layerswithout interlayer connections

Step 5 Constructing the third component of the MKN amonolayer projection of M mp(M) where we project allnodes and edges onto one single layer As the result weobtain mp(M) which consists of one network layer with allthe nodes and edges

Step 6 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the global level

Step 7 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the local level in terms of iden-tifying key concepts on each layer

Step 8 Identifying communities aiming for the justificationof knowledge organization and hierarchical partitioning ofinstructional content into coherent groups of concepts

Step 9 Analyzing four layers in terms of assortativitymixing

Step 10 Analyzing four projected layers in terms of nodeand edge overlapping

Theknowledgedimension-LAYER

Factual

Conceptual

Procedural

Metacognitive

Knowledge dimension [LAYER]

The Cognitive process dimension - WEIGHT

(1) Factual(2) Conceptual(3) Procedural(4) Metacognitive

(2)Understand

(1)Remember

(3)Apply

(4)Analyze

(5)Evaluate

(6)Create

Cognitive process dimension [WEIGHT](1) Remember(2) Understand(3) Apply(4) Analyze(5) Evaluate(6) Create

Knowledge unitDatabase DBMS

Verbdifferentitate

Instructional objectiveStudents will differentiate between terms Database and DBMS

xStudent will differentiatebetween term Database

and DBMS

Figure 1 Construction of edges for the outcome ldquoStudents will differentiate between terms database and DBMSrdquo results with two nodes(knowledge units) database and DBMS and the verb differentiate establishing the edge with weight 4 in cognitive process dimension(analyze) at the layer of factual knowledge

6 Complexity

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

32 Modeling the Network of Concepts Given the fact thatthe network of concepts in itsrsquo simplest definition repre-sents a system of connected parts we assume that theknowledge system of any domain (subject) can be modeledas a complex network of knowledge In the context of theselected domain (course) the concepts are nodes and theirrelationships may be represented as edges that connectknowledge units e edges are directed reflecting the de-pendence of a hierarchical ordering that follows from thenavigational path of learning in which one unit of knowledgeis introduced (acquired) before the other Course designreflects an effective instructional plan aiming to optimizealignment between learning objectives assessments andinstructional activities organized in a specific ordering oftopics (ie crucial concepts of the domain) Hence theassumption is that the network of concepts follows theordering of efficient knowledge acquisition through thelearning process [1ndash4 6 39 49]

Determining learning outcomes start from a higher leveland at first determines the learning outcomes for the studyprogram then for the module and then for the group ofsubjects followed by outcomes of individual subjects whichare finally decomposed into knowledge units [17 59] eplanned outcomes (learning objectives) can be expressed interms of the content (expressed as nouns) and the cognitiveprocess needed (expressed as verbs) In this sense [7] theoriginal Bloomrsquos taxonomy was one-dimensional becausethe categories contained only nouns (objects) and verbs(actions) e verb generally refers to the actions associatedwith the intended cognitive process e object generallydescribes the knowledge students are expected to acquire orconstruct e cognitive process dimension represents acontinuum of increasing cognitive complexitymdashfrom re-member to create [16] Each node is a concept or knowledgeunit and it is a semantic part of a learning objective erelation between two nodes (two knowledge units) isestablished if these two knowledge units appear in the same(common) learning objective (outcome) and vice versa thelearning objective is a relation between two concepts (nodes)concerning the cognitive process and the knowledge di-mensions according to revised Bloomrsquos taxonomy RevisedBloomrsquos taxonomy differentiates between four dimensionsof knowledge factual conceptual procedural or meta-cognitive knowledge [16] According to this taxonomy eachlevel of knowledge corresponds to the level of cognitiveprocess (load) so a student can remember factual or pro-cedural knowledge understand conceptual or metacognitiveknowledge or analyze metacognitive or factual knowledge[15ndash18] Moreover according to the revised version ofBloomrsquos taxonomy there are six dimensions of the cognitiveprocess remembering understanding applying analyzingevaluating and creating

Hence revised Bloomrsquos taxonomy can be represented asa multilayer network where each layer models one di-mension of knowledge To this end we propose a multi-dimensional knowledge network (MKN) formally definedin the previous section

First we define a multilayer network M which consistsof four layers Each layer represents one dimension (the

knowledge level) of Bloomrsquos taxonomy factual conceptualprocedural and metacognitive knowledge In each layernodes represent knowledge units defined according to thelearning outcome It is important to note that one node maybelong to different layers since it can be associated withdifferent learning outcomesmdashhence different levels ofBloomrsquos taxonomy Still it is not required that all nodes(knowledge units) are present on all layers ie someconcepts are for instance present at factual and conceptuallayers and not at procedural and metacognitive oneserefore we opt for modeling of the multilayer and not forthe multiplex network

Nodes are connected between each other in one layer(intralayer edges) and across layers (interlayer edges) Anedge between two nodes (knowledge units) is constructed ifthere is a learning outcome in which the first knowledge unitis connected to the second knowledge unit (in terms ofsequence which follows the order of knowledge acquisitionwithin the domain) e edges are weighted with regard toBloomrsquos cognitive process dimensions and weights of thecorresponding edge depend on the cognitive learning level tobe achieved through the learning process that includes twonodes (knowledge units) Hence remembering receives aweight of 1 understanding 2 applying 3 analyzing 4evaluating 5 and creating 6

Finally we also construct the monolayer projection ofthe multilayer network M Hence a monolayer networkcontains all nodes and edges weighted with respect to thecognitive process dimension and analyzed as a classicalcomplex network providing information about the globalnetwork properties Results of monolayer network analysisrepresent ldquoinitial rawrdquo data that can be used to identify whichtop-ranked (monolayer) nodes according to differences inresults of monolayer and multilayer analysis should bereviewed concerning its knowledge dimension identity inmultilayer and projected components of the MKN Forexample high in-degree nodes in a monolayer networkshould indicate nodes at the factual knowledge dimensionsince it refers to isolated facts and details of concepts at thevery early stage of learning In the continuation themonolayer network is compared to projected layers in orderto quantify the specific properties of each layer is isgrounded in the theory of learning When the units of thetask are being processed simultaneously or when the contenthas a high degree of interactivity among elements the highcognitive load will be imposed on the student even when thenumber of interreacting elements is relatively small As theintrinsic cognitive load is essential for the achievement ofspecific learning goals (understanding of the problemconstruction of higher structures of knowledge and theirflexibility) it must be within the capacity of workingmemory (ie within limits of intrinsic cognitive load)

Figure 1 (adapted from [16]) illustrates an edge con-struction rule for the learning outcome ldquoStudents willdifferentiate between terms database and DBMSrdquo islearning outcome results with two nodes (knowledge units)database and DBMS with a verb differentiate and establishesthe intralayer edge in 4th (analyze) cognitive process di-mension on the factual layer Similarly for the outcome

Complexity 5

ldquoStudent will explain the correlation between 4NF andMultivalued dependencyrdquo We define interlayer edge (1)node-4NF at procedural layer and (2) nodendashmultivalueddependency at conceptual layer with weight 2 (verb un-derstand) It is worth noticing that knowledge units areassumed to lie along a continuum from concrete (factual) toabstract (metacognitive) e conceptual and proceduralcategories overlap in terms of abstractness e verb gen-erally describes the intended cognitive process and theconcept generally describes the knowledge students areexpected to acquire [16]

e process of the construction of the multidimensionalknowledge network MKN with all three componentsM ip(M) andmp(M) and the process of the analysis canbe summarized in the following ten steps Formalization ofthe process in ten steps can serve as a recipe for the transferof the proposed methodology into a new domain

Step 1 Selecting the knowledge domain and defining ahierarchical list of concepts (knowledge units of the in-structional plan for domain) For instance if we decide toconstruct the MKN for the database domain we will defineconcepts like tables relations normal form and index

Step 2 Designing relevant learning outcomes from relationsbetween concepts and aligning them with respect to Bloomrsquostaxonomy of cognition For example from the learningoutcome ldquoStudents will explain the use of database nor-malization as the systematic approach of decomposing tablesrdquowe can derive a relation decomposing (tables and databasenormalization)

Step 3 Constructing the first component of the MKN amultilayer network M again nodes are concepts from the

domain and one node can belong to one or more layersaccording to the learning outcomes that include that concept

Step 4 Constructing the second component of the MKN aninterlayer projection of M where we project the interlayeredges into the target layer as defined in Section 31 As theresult we have ip(M) which consist of four network layerswithout interlayer connections

Step 5 Constructing the third component of the MKN amonolayer projection of M mp(M) where we project allnodes and edges onto one single layer As the result weobtain mp(M) which consists of one network layer with allthe nodes and edges

Step 6 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the global level

Step 7 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the local level in terms of iden-tifying key concepts on each layer

Step 8 Identifying communities aiming for the justificationof knowledge organization and hierarchical partitioning ofinstructional content into coherent groups of concepts

Step 9 Analyzing four layers in terms of assortativitymixing

Step 10 Analyzing four projected layers in terms of nodeand edge overlapping

Theknowledgedimension-LAYER

Factual

Conceptual

Procedural

Metacognitive

Knowledge dimension [LAYER]

The Cognitive process dimension - WEIGHT

(1) Factual(2) Conceptual(3) Procedural(4) Metacognitive

(2)Understand

(1)Remember

(3)Apply

(4)Analyze

(5)Evaluate

(6)Create

Cognitive process dimension [WEIGHT](1) Remember(2) Understand(3) Apply(4) Analyze(5) Evaluate(6) Create

Knowledge unitDatabase DBMS

Verbdifferentitate

Instructional objectiveStudents will differentiate between terms Database and DBMS

xStudent will differentiatebetween term Database

and DBMS

Figure 1 Construction of edges for the outcome ldquoStudents will differentiate between terms database and DBMSrdquo results with two nodes(knowledge units) database and DBMS and the verb differentiate establishing the edge with weight 4 in cognitive process dimension(analyze) at the layer of factual knowledge

6 Complexity

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

ldquoStudent will explain the correlation between 4NF andMultivalued dependencyrdquo We define interlayer edge (1)node-4NF at procedural layer and (2) nodendashmultivalueddependency at conceptual layer with weight 2 (verb un-derstand) It is worth noticing that knowledge units areassumed to lie along a continuum from concrete (factual) toabstract (metacognitive) e conceptual and proceduralcategories overlap in terms of abstractness e verb gen-erally describes the intended cognitive process and theconcept generally describes the knowledge students areexpected to acquire [16]

e process of the construction of the multidimensionalknowledge network MKN with all three componentsM ip(M) andmp(M) and the process of the analysis canbe summarized in the following ten steps Formalization ofthe process in ten steps can serve as a recipe for the transferof the proposed methodology into a new domain

Step 1 Selecting the knowledge domain and defining ahierarchical list of concepts (knowledge units of the in-structional plan for domain) For instance if we decide toconstruct the MKN for the database domain we will defineconcepts like tables relations normal form and index

Step 2 Designing relevant learning outcomes from relationsbetween concepts and aligning them with respect to Bloomrsquostaxonomy of cognition For example from the learningoutcome ldquoStudents will explain the use of database nor-malization as the systematic approach of decomposing tablesrdquowe can derive a relation decomposing (tables and databasenormalization)

Step 3 Constructing the first component of the MKN amultilayer network M again nodes are concepts from the

domain and one node can belong to one or more layersaccording to the learning outcomes that include that concept

Step 4 Constructing the second component of the MKN aninterlayer projection of M where we project the interlayeredges into the target layer as defined in Section 31 As theresult we have ip(M) which consist of four network layerswithout interlayer connections

Step 5 Constructing the third component of the MKN amonolayer projection of M mp(M) where we project allnodes and edges onto one single layer As the result weobtain mp(M) which consists of one network layer with allthe nodes and edges

Step 6 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the global level

Step 7 Analyzing and comparing all network layers definedin ip(M) and mp(M) on the local level in terms of iden-tifying key concepts on each layer

Step 8 Identifying communities aiming for the justificationof knowledge organization and hierarchical partitioning ofinstructional content into coherent groups of concepts

Step 9 Analyzing four layers in terms of assortativitymixing

Step 10 Analyzing four projected layers in terms of nodeand edge overlapping

Theknowledgedimension-LAYER

Factual

Conceptual

Procedural

Metacognitive

Knowledge dimension [LAYER]

The Cognitive process dimension - WEIGHT

(1) Factual(2) Conceptual(3) Procedural(4) Metacognitive

(2)Understand

(1)Remember

(3)Apply

(4)Analyze

(5)Evaluate

(6)Create

Cognitive process dimension [WEIGHT](1) Remember(2) Understand(3) Apply(4) Analyze(5) Evaluate(6) Create

Knowledge unitDatabase DBMS

Verbdifferentitate

Instructional objectiveStudents will differentiate between terms Database and DBMS

xStudent will differentiatebetween term Database

and DBMS

Figure 1 Construction of edges for the outcome ldquoStudents will differentiate between terms database and DBMSrdquo results with two nodes(knowledge units) database and DBMS and the verb differentiate establishing the edge with weight 4 in cognitive process dimension(analyze) at the layer of factual knowledge

6 Complexity

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

4 Multidimensional Knowledge NetworkConstruction for the Learning Outcomes inthe Database Domain

e concepts and their relationships are established fromknowledge units defined in database syllabus standardlyincluded in the computer science bachelor study (steps 1and 2mdashplease note that two authors have a background inteaching relational database subjects and acted as expertsin the domain) e process continues with the con-struction of a multilayer knowledge network for the da-tabase domain (Step 3) First we construct the multilayernetwork M e multilayer network has a factual con-ceptual procedural and metacognitive layer Each node isa unit of knowledge and it is a semantic part of a learningobjective Two nodes are connected if there is a learningoutcome that includes both units e multilayer networkis constructed as directed and weighted network ofconcepts

Next we construct a projection of a multilayer network(Step 4) e projected network is directed and weightedand has factual conceptual procedural and metacognitivelayers e factual layer contains 19 edges25 nodes theconceptual layer 5743 the procedural layer 5042 and themetacognitive layer 2025 edgesnodes Figure 2 presentsfour projected layers of a multilayer network M for thelearning outcomes of the database domain Layer withfactual cognitive knowledge is on the left followed bylayers of conceptual and procedural knowledge while themetacognitive layer is at the rightmost position en weconstruct the monolayer projection (Step 5) again asdirected and weighted network with 59 nodes and 147edges

Next we analyze the projected networks on the globallevel (Step 6) e analysis is based on the quantification ofstandard network measures as defined in SupplementaryMaterial For both projected multilayer and projectedmonolayer networks we calculate average degree averageweighted degree network diameter average path lengthaverage clustering coefficient graph density and the numberof connected components

Global level quantification is followed by local levelquantification of network properties (Step 7)e first aim ofthe analysis is to determine which concepts are the key onesthat stimulate cognitive processes and are of importance foreffective knowledge acquisition According to the resultsobtained in our previous study on keywords extraction[34 35] initially we use degree in-degree and out-degree(also with weighted variants) and proceed with centralitymeasures of closeness betweenness and eccentricity

en we analyze communities (Step 8) aiming for thejustification of knowledge organization and hierarchicalpartitioning of instructional content into coherent groups ofconcepts

Next we compare multilayer characterizations of assor-tativitymixing in terms of Pearson and Spearman correlationsbetween layers for assessing the resemblance and coherencebetween Bloomrsquos knowledge dimensions (Step 9)

And finally we perform the overlapping analysis (Step10) which enables better insights into relatedness of fourprojected layers in terms of node and edge overlapping

All visualizations and network measures are obtainedwith Gephi [60] and MuxViz [30] tools Gephi is open-source software for graph and network analysis whichcomes with a range of layout algorithms [60] MuxViz is afree and open-source package for the analysis and visuali-zation of multilayer networks [61]

5 Results and Discussion

In this section we represent the results of the systematicanalysis of network layers on the global local and meso-scalelevel and provide the results of measuring of correlations andoverlapping between MKN layers e measures have beenselected to reveal crucial structural properties identificationof central concepts in MKN detection of a coherent cluster ofconcepts and quantifying the relationship between differentlevels of the abstraction in the domain Moreover the centralopen questions are to understand the hierarchical depen-dencies along the knowledge dimension and dependencybetween knowledge units

51 Analysis of theMKNNetwork Layers on the Global Localand Meso-Scale Level Initially we present the results ofglobal characterization of the structural properties forprojected multilayer andmonolayer of themultidimensionalknowledge network (MKN) in Table 1 All global measures(average degree average weighted degree network diameteraverage path length average clustering coefficient andnetwork density) are calculated for the largest connectedcomponent Note that equations and explanations of allthese network measures are given in the Supplementarymaterial

Global network measures enable only the coarse dif-ferentiation between layers Still we can notice that theconceptual and procedural layers exhibit similar proper-tiesmdashhigher values of average degree average weighteddegree diameter average path length and graph density incomparison to factual and metacognitive layers Moreoverconceptual and procedural layers exhibit properties that arecloser to the monolayer network than to the other two layerse average clustering coefficient has low values for all fourlayers (for the metacognitive layer the clustering coefficientis so low that it was not possible to calculate it) emonolayer network has a higher clustering coefficient thatmay indicate a higher cognitive load during the learning ofnew and yet strongly interwoven concepts which supportsthe initial premise of balancing the cognitive load duringlearning staring from concrete to abstract and better-interconnected knowledge units of the domain

Moreover factual and metacognitive layers have morethan double the number of components compared toconceptual and procedural layers which reflects in thehigher values of global network measures e number ofconnected components is the highest in the factual layer andtogether with smaller network diameter is the reflection of

Complexity 7

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

high fragmentation of basic constituents needed forknowledge acquisition

e primary aim of any centrality measure is the rankingof the nodes for producing an ordered list of the nodesaccording to their relevance in the structure [19 21 47 48 52]e high degree centrality is inherent for hub nodes in thestudied case the degree reflects how influential (central) is aconcept for the process of knowledge acquisition (Table 2)

At the monolayer according to the unweighted andweighted variants we obtain one shared concept of tablesand at a procedural layer the concept of database nor-malization which is the core procedure during the designand construction of databases and at a metacognitive layerwe obtain the concepts of database_scheme (unweighted)and query (for weighted variant) is makes a senseknowing that querying is the primary programming ab-straction of the relational databases and database_schememakes a blueprint of the database construction

Still despite different rankings of the top concepts wehave received the valuable set of concepts characterized bythe rich content ese concepts acquire knowledge of abroader scope of concepts (supported by many previousnodes) and result in a higher cognitive process dimensionNodes with low in-degree represent concepts which can bestarting points of the navigational path through the contentand vice versa nodes with high out-degree correspond toconcepts with learning outcomes of a higher level of thecognitive dimensions For instance it can be noticed thatconcepts Table and Database normalization are detected ashubs and are directly linked with the level of cognitive load

in an interactive learning setting which can be cognitivelychallenging for a novice learner because of a high level oftransitivity between knowledge dimensions Hence it wouldbe advisable to plan the acquisition of these concepts earlierrather than later in the instruction (navigational) plan

Next we perform an analysis of degree values for the top20 ranked nodes inMKN as reported in Figure 3 It is evidentthat the node degree measure acts similarly in all layers andholds similar for in-degree out-degree and strength as wellNode degree in MKN layers exhibits similar results as in themonolayer network Concepts (nodes) labeled as TableDatabase normalization Attributes and Database schemehave high values of degree centrality indicating that thosenodes require additional effort in cognitive processing at acertain level of knowledge (as already noticed above) esenodes are supported with several lower-level nodes indi-cating a more complex knowledge structure and necessity ofthe existence of studentrsquos prior knowledge Considering therole of network structure and different knowledge dimen-sions as relationship types the influence of degree can besignificant in the form of interplay between the cognitiveprocess dimension knowledge type and instructionHowever since the node degree in a multilayer network is avector aggregation of measures could indicate which nodesare more influential and how are related among differentlayers

Figure 4 visualizes nodes of the monolayer network indifferent colors according to its degree values Similar vi-sualizations for closeness centrality betweenness and ec-centricity in a monolayer network are reported in

Table 1 Values of average degree average weighted degree network diameter average path length average clustering coefficient graphdensity and the number of connected components WS (weaklystrongly) for the projection layers of ip(M) and mp(M)

Layer Averagedegree

Averageweighteddegree

Networkdiameter

Averagepathlength

Averageclusteringcoefficient

Networkdensity

Connectedcomponents WS

Factual 0322 0881 3 132 0014 0006 4159Conceptual 0966 3068 8 2608 0017 0017 1759Procedural 0847 3119 6 2769 0015 0015 2056Metacognitive 0339 1220 3 1433 mdash 0006 3959Monolayer 2441 8288 10 4051 0096 0042 123

Relational Algebra

Cardinality

Database Management System

Funtional Dependencies

Business Organisation

Data

Information

Data Integrity Database Scheme

Relational Model

Edgar Codd

Relation

Tables

Database Normalisation

Database

Other Model

Entity

Relational database

Constraints

Data DomainColumns

Attributes

Referential Integrity

Foreign Key

RowsSuperkey

Candidate Key

Cartesian product

Factual Conceptual Procedural Metacognitive

Business Organisation

Information System

Data

Database

Database SchemeTables

Relational Model

Logical Model

Relational ModelPhisical Model

Edgar Codd

RelationDerived Relation

View

Primary Key

Foreign Key

Relational Database

SQL

Entity

ERD

Relationships

Attributes

TablesRows

Columns

Cardinality Ratio

Relationships

Constraints

Referential IntegrityEntity Integrity

Cardinality

Data Integrity

Database Normalisation

1NF2NF

3NF

BCNF

Superkey

Candidate Key

Functional Dependency

4NF

Database Scheme

View

Databaswe

Edgar Codd

Database Normalisation

Fundamental Relation

Join

Entity

Relational Database

Logical Model

Query

Relational Algebra

TablesColumns

Relationships

Entity Integrity Primary Key

Foreign Key

2NF

BCNF

4NF

1NF3NF

Multivalued Dependency

Union

DifferenceIntersection

Cartesian ProductProjection

Selection

Join

ERD

Entity

Database Index

Database IntegritySuperkey

Attributes

Database Scheme

Constraints

Relational Database

Database Normalisation

Entity

Primary Key

Foreign Key

TablesQuery

Relationships

2NF

BCNF

Attributes

System

Business Organisation

Information System

Database Managament System

Database Scheme

Data Domain

Referential Integrity

Superkey

Rows

Functional Dependency

Relation

Relational Algebra

Figure 2 Network representation of four projected layers in a multidimensional knowledge network (MKN) for the learning outcomes ofthe database domain

8 Complexity

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

Supplementary Materials e figure suggests that themonolayer network exhibits some hierarchical propertiesthat may be analyzed on the meso-scale level

Analysis of weighted degree (weighted in-degree andweighted out-degree) does not show substantial differencesbetween the top-ranked nodes across layers is implies

Table 2 Top five highly ranked nodes according to the degree (dci) in-degree dcini and out-degree dcouti values in a monolayer network

Node dci Node dcini Node dcouti

Monolayer

Tables 14 Primary_key 10 Relational_algebra 9Database_normalization 12 Attributes 9 Database_normalization 8

Database_scheme 11 Tables 8 Database_scheme 7Relational_algebra 11 Relation 7 Entity 7

Primary_key 10 Rows 7 Tables 6

Factual

Database_scheme 3 Database_scheme 3 Business_organisation 3Tables 3 Tables 3 Data 2Data 3 Information 2 Constraints 1

Business_organisation 3 Relational_database 2 Database 1Database 2 Data 1 Referential_integrity 1

Conceptual

Attributes 7 Attributes 6 Database 5Database 6 Rows 5 Tables 3Tables 6 Primary_key 5 Foreign_key 3

Columns 6 Columns 4 SQL 3Foreign_key 5 Data_domain 3 Referential_integrity 3

Procedural

Database_normalization 11 Relation 5 Database_normalization 7Relational_algebra 7 Database_normalization 4 Relational_algebra 7

3NF 6 Primary_key 3 3NF 5Relation 6 Entity 2 Entity 2Entity 4 Tables 2 Tables 2

Metacognitive

Database_scheme 3 Foreign_key 3 Database_scheme 2Foreign_key 3 Attributes 2 2NF 2

2NF 2 Primary_key 2 Entity 2Attributes 2 Rows 2 Selection 2Constraints 2 Database_scheme 1 Constraints 1

1NF2NF3NF4NF

AttributesBCBF

Business_organisationCandidate_key

CardinalityCardinality_RatioCartesian_Product

ColumnsConceptualConstraints

DataData_domainData_integrity

DatabaseDatabase_index

DBMS21000 50 100 150 200

ConceptualFactualMonolayer

MetacognitiveProcedural

Figure 3 Diagnostic analysis of degree measure in MKN shows relations across layers

Complexity 9

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

that although the same nodes appear at different knowledgelevels their cognitive complexity can vary due to theknowledge dimension overlap In practice these overlapladders of the cognitive domain and the knowledge dimen-sion do have limits and need interpretations e illustrativeexample is ldquoIs it better for a student to have achieved factualknowledge of creating rather than metacognitive knowledgeof rememberingrdquo ere is no single answer since it differsdepending on the teaching style and subject area

Still the top degree lists (concepts) differ substantiallymeaning that the identification of the most essential (highlyranked) concepts highly depends on the variant of the degreecentrality measure with in- and out-variants and usedweighting is is an indication that for better identificationof the most influential concepts we should opt for moresophisticated insights so first we proceed with the quan-tification of centrality measures (Table 3)

Table 4 presents the values of closeness centrality be-tweenness centrality and eccentricity in a monolayer net-work Closeness centrality quantifies how close a node is toall other nodes in the network the smaller the total distancefrom a node v to all other nodes the more important thenode v [21] According to the closeness centrality valuesentity integrity is a top-ranked node followed by four nodeswith values of the same range is implies that these fourconcepts should be considered as possible starting points in alearning navigation path since their closeness values cor-respond with high degree values Nodes that take a startingpoint role in learning navigational paths are crucial foreffective knowledge acquisition e research reported in

[46] also emphasizes the importance of the closeness cen-trality since it operationalizes the structural relevance in theknowledge representation Note that all nodes with closenesscentrality equal to 1 refer to nodes that belong to smalldisconnected components usually composed of two nodeswhich also indicates the fragmentation of knowledge

Nodes with high betweenness in Table 4 are Databasenormalization and Tables followed by Constraints and Re-lational algebra It is known that these nodes take a bridgingrole in the network ie they are in charge of the informationflowmdasheither for describing other knowledge units (the in-coming edges) or for influencing other knowledge units (theoutgoing edges) ese nodes are of high importance andserve as the glue in the knowledge representation model

e eccentricity aims to determine a node that mini-mizes the maximum distance to any other node in the graphIn other words eccentricity quantifies the distance betweenthe concepts Top-ranked Selection Attributes Derived re-lation and System represent core concepts needed for ac-quiring more complex knowledge e high value ofeccentricity indicates that concept could be essential inminimizing learning effort while acquiring more complexconcepts Except for the most essential concept of Databaseat the factual layer the centrality analysis was failing toreveal better structural ordering or sequencing of the con-cepts during learning

Detecting communities in complex networks is of par-ticular interest when identifying nodes that share propertiesand dynamics [21] In this research we apply the Louvainalgorithm [62] for community detection and Figure 5 shows

1NF

2NF

3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relation

Difference

Edgar_Codd

Entity

Entity_integrity

Entityrelationship_diagram

External

Foreign_key

Functional_dependencyFundamental_relation

Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_database

Relational_model

Relationships

Rows

Selection

SQL

Superkey

System

Tables

Union

View

Figure 4 e visualization of the monolayer network according to the degree Nodes with high degree values are darker while nodes withlower degree values are lighter colored

10 Complexity

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

Table 3 Top five highly ranked nodes according to the weighted degree (oij) weighted in-degree (oinij ) and weighted out-degree (ooutij ) in aweighted monolayer network

Node oij Node oinij Node ooutij

Monolayer

Tables 62 Tables 34 Database_normalization 41Database_normalization 57 Primary_key 32 Tables 28

Relation 41 Relation 31 Entity 25Foreign_key 38 Attributes 27 Database_scheme 23

Entity 38 Columns 24 Database 23

Factual

Database_scheme 12 Database_scheme 12 Data 6Tables 10 Tables 10 Business_organisation 6Data 8 Information 6 Database 4

Business_organisation 6 Database_ManagementSystem 4 Candidate_key 4Database 6 Relationships 4 Cardinality 4

Conceptual

Tables 29 Columns 18 Database 19Columns 24 Primary_key 18 Tables 14Database 23 Attributes 16 Referential_integrity 12

Foreign_key 20 Tables 15 1NF 12Attributes 18 Rows 12 Database_scheme 11

Procedural

Database_normalization 52 Relation 22 Database_normalization 36Relation 26 Database_normalization 16 Relational_algebra 143NF 17 Tables 9 3NF 11Tables 17 2NF 8 Projection 11Entity 16 Join 8 Entity 9

Metacognitive

Foreign_key 12 Foreign_key 12 2NF 7Query 11 Query 6 Business_organisation 6

Relational_algebra 10 System 6 Entity 6Functional_dependency 9 Attributes 6 Tables 6

Constraints 8 Primary_key 6 Query 5

Table 4 Top five highly ranked nodes (concepts) according to the closeness centrality (cci) betweenness centrality (bci) and eccentricity(Ceec) in a monolayer network

Node cci Node bci Node Ceec

Monolayer

Entity integrity 10 Database normalization 806546 Selection 100Database scheme 0362 Tables 624485 Attributes 90

Database normalization 0359 Constraints 5881 Derived_relation 90Tables 0354 Relational_algebra 507413 System 80

Relational algebra 0336 Query 490413 Business_organisation 80

Factual

Database 10 Database 20 Business_organisation 30Referential_integrity 10 Referential_integrity 20 Functional_dependency 30

Candidate_key 10 Data 20 Data 20Cardinality 10 Constraints 20 Constraints 20

Cardinality_Ratio 10 Candidate_key 00 Database 10

Conceptual

View 10 Data_domain 640 Information_System 80Entity_integrity 10 Constraints 550 Database 70

Cartesian_product 10 Attributes 395 Relational_model 60Relational_algebra 10 Columns 285 Database_scheme 60

SQL 08 Entity 250 Entity 50

Procedural

Database_scheme 10 Database_normalization 1560 Relational_algebra 60Relationships 10 3NF 495 3NF 50

Fundamental_relation 10 Tables 460 Cartesian_product 502NF 10 Entity 430 Projection 50BCBF 10 Relational_database 360 Database_index 50

Metacognitive

Entity 10 Database_scheme 40 Tables 30Relational_algebra 10 Constraints 30 Database_normalization 30

Functional_dependency 10 Relational_algebra 20 Constraints 20Referential_integrity 10 Query 20 Query 20Database_scheme 10 Functional_dependency 10 2NF 20

Complexity 11

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

the results of community detection in the monolayer MKNnetwork

ere are six communities in the monolayer networkwhile the whole MKN is dispersed into 22 communitiesAccording to the community structure in the monolayernetwork (Figure 5) we assume that closely related learningconcepts belong to the same community forming a coherentgroup of knowledge units For example concepts DatabaseDatabase Management System Logical Model and PhysicalModel while Relational Algebra Query and Union belong toanother community Still the community structure is notideally discovered sinceDatabase Normalization 2NF 3NFand 4NF are in the same community while 1NF is in theother although semantically belongs to the same Howeverit seems that community structure provides a good insightinto how concepts are clustered into more complex unitsand is of benefit for the detection of coherent groups ofconcepts which can be organized in the same learning units(lectures)

52 Analysis of Correlations and Overlapping between MKNLayers A standard way to quantify the presence of inter-layer degree correlations is to calculate Pearsonrsquos andSpearmanrsquos interlayer correlation coefficients for indicationof how degree sequences of two layers are correlated [10]Networks that are degree assortative or degree disassortativehave higher information content than networks that aredegree nonassortative [61] e values of assortativity areobtained as Pearson and Spearman pairwise correlationcoefficients for multilayer networks as shown in Tables 5and 6 respectively Each of those two coefficients exposesslightly different behaviors More precisely the values of

Pearsonrsquos correlation coefficient indicate disassortativity offactual conceptual and procedural knowledge e reasonstems from the very type of knowledge they represent andtheir differentiation factual knowledge represents specificbits of information conceptual knowledge includes morecomplex organized knowledge (schemas models and the-ories) and procedural knowledge reflects knowledge ofldquoprocessesrdquo In order to acquire knowledge of a higher levelldquothe deeper understandingrdquo (higher cognitive process di-mension) of low-level units is crucial hence the dis-assortativity between layers

Layers of the multidimensional knowledge network(MKN) stem from four knowledge dimensions ey aresequenced from the detailed factual knowledge in the factuallayer to the abstract metacognitive knowledge in the

1NF

2NF 3NF

4NF

Attributes

BCBF

Business_organisation

Candidate_key

Cardinality

Cardinality_Ratio

Cartesian_product

Columns

Conceptual

Constraints

Data

Data_domain

Data_integrity

Database

Database_index

Database_ManagementSystem

Database_normalization

Database_scheme

Derived_relationDifference

Edgar_Codd

Entity

Entity_integrityEntityrelationship_diagram

External

Foreign_key

Functional_dependency

Fundamental_relation Information

Information_System

Internal

Intersection

Join

Logical_model

Multivalued_dependency

Navigational_operators

Other_model

Physical_model

Primary_key

Projection

Query

Referential_integrity

Relation

Relational_algebra

Relational_databaseRelational_model

RelationshipsRows

Selection

SQL

Superkey

System

Tables

UnionView

Figure 5 e structure of six communities in a monolayer MKN network

Table 5 Interlayer assortativity Pearsonrsquos correlation coefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0175 0111Conceptual 0096 1 minus 0033 0281Procedural minus 0175 minus 0033 1 0155Metacognitive 0111 0281 0155 1

Table 6 Interlayer assortativity Spearmanrsquos correlationcoefficients

Layer Factual Conceptual Procedural MetacognitiveFactual 1 0096 minus 0241 0085Conceptual 0042 1 0023 0226Procedural minus 0241 0023 1 016Metacognitive 0085 0226 016 1

12 Complexity

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

metacognitive layer In some cases edges of the layers arenot mutually exclusive which can be indicated by the nodesoverlapping values quantified by correlations between thedegrees of the same node at different layers On the otherside edge overlapping also suggests that the relation betweentwo nodes might be shared by more than one layer whichcould indicate redundancy of information input In Tables 7and 8 we show the percentage of overlapping for nodes andedges respectively e results confirm the organization ofthe layers according to Bloomrsquos taxonomy is highly advisablesince edge overlapping between conceptual and meta-cognitive is minimal (below 32) while the node overlapcan go to the high 525

Table 9 shows the Frobenious distance used to quantifylayer distance in terms of paths e highest values of Fro-benious distance are achieved formetacognitivefactual layersrsquopairs which are themost distant layersis is an indication ofhigher cognitive effort while traversing from factual to met-acognitive layermdasha larger knowledge ldquojumprdquo during learningIn other words learning ldquofragments of informationrdquo (factualknowledge) requires establishing connections between frag-mented facts and applying them in new situations whichrequire a higher level of studentsrsquo cognition (metacognitiveknowledge) One of the directions for the reduction of thecomplexity of the proposed MKN model especially in lessdistant layers (factual conceptual and procedural) can beachieved by structural reducibility proposed in [63]

Still this remains an open challenge in future researchplans e results obtained from MKN analysis can shedlight on the causes of increased cognitive demands indicatevulnerabilities in the knowledge (more specific and identifyknowledge units that require modification of instructionalstrategies) and consequently guide the design and opti-mization of learning outcomes erefore the relationshipbetween the structure of information and external repre-sentation of knowledge should be pursuit with the identi-fication of concepts that play the key role of ldquobasic buildingblocksrdquomdashhigh in-degree nodes in monolayer network andcomparison with results generated from projected MKN thedetection of concepts with high betweenness to reveal theglue concepts of the domain represented in MKN and withhigh value of eccentricity to detect concepts that could beessential in minimizing the learning effort while acquiringmore complex concepts the identification of clusters ofconceptsmdashcommunities can lead to better planning of thelectures and understanding of the knowledge gap betweencognitive layers can reduce the overload burden from thenovice in the field

In general the level of knowledge dimension that isselected for external representation can influence the

adoption of information including the activation of thecognitive process With an inadequate representationalmodel the learning process can be impeded by avoiding thecoherent knowledge units which in turn can lead tostructural vulnerability of the domain model during theknowledge acquisition

6 Conclusion

In the discipline of technology-enabled learning in generalone of the most challenging problems is the study of theformation and representation of knowledge structuresduring learning e goal is often accomplished byexpressing the expertrsquos (tutor) knowledge which is pre-sumed to be well organized coherent and consisting of richexpertise about the subject [52] Recent cognitively orientedresearch on learning implicates that there is a close rela-tionship between knowledge structure and its content ininteraction with the cognitive architecture and learnerrsquosability to process that information Hence this supports theidea that the knowledge system is an interwoven cohesivenetwork that differs according to onersquos mental model priorknowledge and preferences Understanding the structure ofscientific knowledge often refers to topological featureswhere coherence and contingency have a high correlationwith crucial concepts and their interconnectedness usKoponen and Nousiainen in [1] emphasized to make co-herence a clear and useful notion and to design educationalsolutions there must be a chain of connections from co-herence to the operational measures used to characterizeknowledge networks Siew in [50] used macrolevel networkmeasures to quantify the structure of a monolayer networkof concept for the domain of psychology trying to identifyand prioritize the ldquogluerdquo concepts in the network andshowing that internal representation of the studentsrsquoknowledge map can be an indicator of expected performanceand specifically inherent to various learning styles Hencethe progress can be achieved by better personalization of thecontent In this study we have reached a step further andproposed a multilayered organization of external knowledgeas a representation modeling alternative To this end we

Table 7 e percentage overlapping of nodes in the MKNnetwork

Layer Factual Conceptual Procedural MetacognitiveFactual 100 305 237 203Conceptual 100 525 322Procedural 100 305Metacognitive 100

Table 8e percentage overlapping of edges in theMKN network

Layer Factual Conceptual Procedural MetacognitiveFactual 100 0 0 0Conceptual 100 0 32Procedural 100 16Metacognitive 100

Table 9 Frobenious distance calculated between all pairs of nodesin each layer separately

Layer Factual Conceptual Procedural MetacognitiveFactual mdash 0 0151 0897Conceptual mdash 0144 0019Procedural mdash 0192Metacognitive mdash

Complexity 13

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

proposed modeling according to the revised Bloomrsquos tax-onomy in a multidimensional knowledge network (MKN)

Recently some advances toward shedding more light onthis ambitious pursuit paw the way of the future researchquests Several studies indicate that multilayer or multiplexrepresentations are adequate modeling approaches for thecognitive representation models Stella in [45] suggests thatglobal and multilevel representation of the mental lexiconfor acquiring vocabulary in the early stage of learninglanguage competencies better models and quantifies the flowof information especially emphasizing the importance ofcloseness centrality for spreading of activation patternsHence studying the dynamics of knowledge acquisition inan e-learning system can be of utmost importance deArruda et al in [64] already revealed that using the true self-avoiding random walk can efficiently model the dynamics ofthe knowledge acquisition which is specifically placed at thecore of the network

Guided by findings in studies of concept and cognitivenetworks [1ndash4 6 8 13 14 22 36 37 39 4146 49 50 52 59 64 65] we have applied the conceptmapping method to represent a knowledge system of theDatabase domain as the complex network Specifically wepropose a multidimensional knowledge network (MKN)based on themultilayer network where each layer constitutesfactual conceptual procedural or metacognitive knowl-edge In the layer nodes are concepts or knowledge unitsand the edges are weighted with regard to the revisedBloomrsquos cognitive learning level Additionally we intro-duced two projections of M the interlayer projection andmonolayer projectionse proposed interlayer projection iscontrasted with monolayer projection by comparing char-acterizations of the centrality measures degree centralitycloseness centrality betweenness centrality and eccentricitye study revealed indications of how concepts supportedwith the higher number of previously introduced conceptshave a dominant role in knowledge acquisition from a viewof knowledge structure and content is can be of use forbetter planning and organization of the content in thee-learning system uniquely when equipped together withcontinuous evaluation of studentsrsquo progress which can leadto a better adaptation of the system

Moreover obtained results indicate that MKN is theadequate model to study the importance and groupings ofthe concept aiming toward the more efficient organizationof concepts Our study indicates that the principles of theknowledge organization of concepts enabling the detectionof ones that are candidates for entry points of the naviga-tional paths or the ones which acquire a higher level of thecognitive domain hence the ones that are crucial for re-ducing or increasing a cognitive load during learning esefeatures were also recognized by [51] whose key conceptswere central from the viewpoint of the richness of subjectcontent and correlated with learning gains as well Animportant but as of yet incompletely resolved issue is howthe construction of knowledge network could directly in-fluence the efficiency of navigation paths during learningespecially with regard to acquiring knowledge on the highestlevel of complexity

Considering the importance of the study of the complexnetwork for understanding and simulating cognitive pro-cesses the correlations between knowledge dimensions wereinvestigated Although there is a substantial similarity re-garding key concepts in the monolayer network theassortativity and shortest path distance values could indicateelements of knowledge structures that learners can activaterapidly and apply to improve the studentsrsquo knowledge iscertainly depends on the mental models of individuals theirprior knowledge cognitive functioning (interaction ofworkingmemory and long-termmemory systems) as well asnavigational patterns learning styles and preferences eobtained results suggest that careful modeling brings dif-ferent perspectives onto modeling of the external knowledgeand results in a more comprehensive understanding of howthe knowledge should be organized across different levels ofcognitive load is is in line with findings reported in [36]where the authors emphasized the need for multiplex overmonolayer representation for early language learning since itallows for quantification of distinct phases in the process

e analysis of the factual layer provides quantificationfor the well-studied fragmentation problem (lack of estab-lished connections between facts into a more extensivesystem of domain knowledge [15 38] by calculating thenetwork measures) is is a step toward bridging the gapbetween the fragmentation of factual knowledge and a morein-depth level or integration or systematic organization ofdomain knowledge in adaptive e-learning systems Apartfrom the knowledge of different strategies and knowledge ofcognitive tasks metacognitive knowledge also includes aself-awareness of proficiency in the domain Hence withoutthe self-awareness of lack at any of factual conceptual orprocedural layers it is unlikely that students will make anyprogress in acquiring or constructing additional knowledge

In this study we reach for a better representation of theexternal knowledge resulting in the more comprehensiveinsights on how the knowledge should be organized acrossdifferent levels of cognitive load To this end we proposedmodeling according to the revised Bloomrsquos taxonomy Wehave confirmed that careful modeling shed different per-spectives onto modeling of the external knowledge repre-sentation Still there remains the open challenge of how weshould approach to modeling and quantification of internalstudentrsquos model of the domain

Despite the promising results of the MKN there are stillmany open research questions which we plan to address infuture research Specifically this includes the application ofthe obtained results into the e-learning system which adaptsto studentrsquos current level of knowledge and suggests the bestnavigational path through the learning content and quan-tification of the studentrsquos progress Additionally we shouldopt for the reduction of the complexity of the proposedMKN model where structural reducibility proposed in [63]can serve as the starting point

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

14 Complexity

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

Conflicts of Interest

e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

is work has been supported in part by the University ofRijeka under the project numbers uniri-drustv-18-20 anduniri-drustv-18-38

Supplementary Materials

Supplementary Text 1 network structure analysis on theglobal and local level Supplementary Figure S1 the visu-alization of the monolayer network after applying ForceAtlas layout algorithm Supplementary Figure S2 visuali-zation of closeness centrality in a monolayer network Nodeswith high closeness values are colored darker whilst thosewith lower values are lighter Supplementary Figure S3visualization of betweenness centrality in a monolayernetwork Nodes with high betweenness values are darkerwhilst those with lower values are lighter SupplementaryFigure S4 visualization of eccentricity in a monolayernetwork Nodes with high eccentricity values are darkerblue whilst those with lower values are brown (Supple-mentary Materials)

References

[1] I T Koponen and M Pehkonen ldquoCoherent knowledgestructures of physics represented as concept networks inteacher educationrdquo Science amp Education vol 19 no 3pp 259ndash282 2010

[2] I T Koponen and M Nousiainen ldquoModelling studentsrsquoknowledge organisation genealogical conceptual networksrdquoPhysica A Statistical Mechanics and Its Applications vol 495pp 405ndash417 2018

[3] I Koponen and M Nousiainen ldquoPre-service physics teachersrsquounderstanding of the relational structure of physics conceptsorganising subject contents for purposes of teachingrdquo In-ternational Journal of Science and Mathematics Educationvol 11 no 2 pp 325ndash357 2013

[4] I T Koponen T Kokkonen and M Nousiainen ldquoDynamicsystems view of learning a three-tiered theory in physicsrobust learning outcomes as attractorsrdquo Complexity vol 21no S2 pp 259ndash267 2016

[5] A Cantildeas J Coffey M Carnot and P J Feltovich ldquoAsummary of literature pertaining to the use of conceptmapping techniques and technologies for education andperformance supportrdquo Technical Report Chief of NavalEducation and Training Pensacola FL USA 2003

[6] I T Koponen M Nousiainen and M Nousiainen ldquoConceptnetworks in learning finding key concepts in learnersrsquo rep-resentations of the interlinked structure of scientific knowl-edgerdquo Journal of Complex Networks vol 2 no 2 pp 187ndash2022014

[7] B Bloom M Englehart E Furst et al Taxonomy of Edu-cational Objectives Ce Classification of Educational GoalsrdquoHandbook I Cognitive Domain Longmans Green amp Co NewYork NY USA 1956

[8] C S Q Siew D U Wulff N M Beckage and Y N KenettldquoCognitive network science a review of research on cognitionthrough the lens of network representations processes anddynamicsrdquo Complexity vol 2019 Article ID 210842324 pages 2019

[9] M Nadrljanski ETH Vukic and ETH Nadrljanski ldquoMulti-agentsystems in E-Learningrdquo in Proceedings of the 41st Interna-tional Convention on Information and CommunicationTechnology Electronics and MicroelectronicsmdashMIPRO Opa-tija Croatia May 2018

[10] R Noldus and P Van Mieghem ldquoAssortativity in complexnetworksrdquo Journal of Complex Networks vol 3 no 4pp 507ndash542 2015

[11] B A Schwendimann ldquoConcept maps as versatile tools tointegrate complex ideas from kindergarten to higher andprofessional educationrdquo Knowledge Management ampE-Learning An International Journal vol 7 no 1 pp 73ndash992015

[12] C S Q Siew andM S Vitevitch ldquoe phonographic languagenetwork using network science to investigate the phono-logical and orthographic similarity structure of languagerdquoJournal of Experimental Psychology General vol 148 no 3pp 475ndash500 2019

[13] M Stella ldquoCohort and rhyme priming emerge from themultiplex network structure of the mental lexiconrdquo Com-plexity vol 2018 Article ID 6438702 14 pages 2018

[14] M Stella N M Beckage M Brede and M De DomenicoldquoMultiplex model of mental lexicon reveals explosive learningin humansrdquo Scientific Reports vol 8 no 1 p 2259 2018

[15] C Bereiter and M Scardamalia ldquoBeyond bloomrsquos taxonomyrethinking knowledge for the knowledge agerdquo in Interna-tional Handbook of Educational Change A HargreavesA Lieberman M Fullan et al Eds pp 675ndash692 KluwerAcademic Publishers London UK 1998

[16] L W Anderson D R Krathwohl P W Airasian et al ATaxonomy for Learning Teaching and Assessing A Revision ofBloomrsquos Taxonomy of Educational Objectives Pearson NewYork NY USA 2001

[17] A Amer ldquoReflections on Bloomrsquos revised taxonomyrdquo Elec-tronic Journal of Research in Educational Psychology vol 4no 1 pp 213ndash230 2006

[18] F Radmehr and M Drake ldquoRevised bloomrsquos taxonomy andintegral calculus unpacking the knowledge dimensionrdquo In-ternational Journal of Mathematical Education in Science andTechnology vol 48 no 8 pp 1206ndash1224 2017

[19] A-L Barabasi and M Posfai Network Science CambridgeUniversity Press Cambridge UK 2016

[20] F Battiston V Nicosia and V Latora ldquoMetrics for theanalysis of multiplex networksrdquo Physical Review E vol 89Article ID 032804 2014

[21] L da F Costa F A Rodrigues G Travieso and P R VillasldquoCharacterization of complex networks a survey of mea-surementsrdquo Advances in Physics vol 56 no 1 pp 167ndash2422007

[22] A Baronchelli R Ferrer-I-Cancho R Pastor-SatorrasN Chater and M H Christiansen ldquoNetworks in cognitivesciencerdquo Trends in Cognitive Sciences vol 17 no 7pp 348ndash360 2013

[23] A Sole A Arenas and S Gomez ldquoEffect of shortest pathmultiplicity on congestion of multiplex networksrdquo NewJournal of Physics vol 21 no 3 Article ID 035003 2019

[24] J Cardillo M Gomez-Gardentildees M Zanin et al ldquoEmergenceof network features from multiplexityrdquo Scientific Reportsvol 3 no 1 p 1344 2013

Complexity 15

[25] J F Donges H C H Schultz N Marwan Y Zou andJ Kurths ldquoInvestigating the topology of interacting net-worksrdquo Ce European Physical Journal B vol 84 no 4pp 635ndash651 2011

[26] J Gao D Li and S Havlin ldquoFrom a single network to anetwork of networksrdquo National Science Review vol 1 no 3pp 346ndash356 2014

[27] L da Fontoura Costa ldquoLearning about knowledge a complexnetwork approachrdquo Physical Review E vol 74 no 2 ArticleID 026103 2006

[28] M Berlingerio M Coscia F Giannotti A Monreale andD Pedreschi ldquoFoundations of multidimensional networkanalysisrdquo in Proceedings of the Advances in Social NetworksAnalysis and Mining (ASONAM) vol 485ndash489 KaohsiungTaiwan August 2011

[29] M De Domenico A Sole- Ribalta E Cozzo et al ldquoMathe-matical formulation of multilayer networksrdquo Physical ReviewX vol 3 no 4 pp 041022ndash041037 2013

[30] M De Domenico M A Porter and A Arenas ldquoMuxViz atool for multilayer analysis and visualization of networksrdquoJournal of Complex Networks vol 3 no 2 pp 159ndash176 2015

[31] M Kivela A Arenas M Barthelemy J P GleesonY Moreno and M A Porter ldquoMultilayer networksrdquo Journalof Complex Networks vol 2 no 3 pp 203ndash271 2014

[32] M Kurant and P iran ldquoLayered complex networksrdquoPhysical Review Letters vol 96 no 13 pp 138701ndash1387052006

[33] S O Tergan ldquoDigital concept maps for managing knowledgeand informationrdquo in Knowledge and Information Visualiza-tion pp 185ndash204 Springer Berlin Germany 2005

[34] S Boccaletti G Bianconi R Criado et al ldquoe structure anddynamics of multilayer networksrdquo Physics Reports vol 544no 1 pp 1ndash122 2014

[35] S Martincic-Ipsic D Margan and A Mestrovic ldquoMultilayernetwork of language a unified framework for structuralanalysis of linguistic subsystemsrdquo Physica A Statistical Me-chanics and Its Applications vol 457 pp 117ndash128 2016

[36] M Stella N M Beckage and M Brede ldquoMultiplex lexicalnetworks reveal patterns in early word acquisition in chil-drenrdquo Scientic Reports vol 7 no 1 p 46730 2017

[37] G Rosell-Tarrago E Cozzo and A Dıaz-Guilera ldquoA complexnetwork framework to model cognition unveiling correlationstructures from connectivityrdquo Complexity vol 2018 ArticleID 1918753 19 pages 2018

[38] J D Branslord A L Brown and R R Cocking How PeopleLeam Brain Mind Experience and school National AcademyPress Washington DC USA 1999

[39] I T Koponen and M Pehkonen ldquoEntropy and energy incharacterizing the organization of concept maps in learningsciencerdquo Entropy vol 12 no 7 pp 1653ndash1672 2010

[40] F Safayeni N Derbentseva and A J Cantildeas ldquoA theoreticalnote on concepts and the need for cyclic concept mapsrdquoJournal of Research in Science Teaching vol 42 no 7pp 741ndash766 2005

[41] NM Beckage and E Colunga ldquoLanguage networks as modelsof cognition understanding cognition through languagerdquo inTowards a Ceoretical Framework for Analysing ComplexLinguistic Networks Understanding Complex SystemsA Mehler A Lucking S Banisch et al Eds pp 3ndash28Springer Berlin Germany 2016

[42] I Gurevych ldquoUsing the structure of a conceptual network incomputing semantic relatednessrdquo in Natural LanguageProcessingmdashIJCNLP 2005 Lecture Notes in Computer Science

R Dale KF Wong J Su et al Eds Vol 3651 SpringerBerlin Germany 2005

[43] J B Batista and L F Costa ldquoKnowledge acquisition bynetworks of interacting agents in the presence of observationerrorsrdquo Physical Review E vol 82 no 1 Article ID 0161032010

[44] M S Vitevich and N Castro ldquoUsing network science in thelanguage and clinicrdquo International Journal of Speech-Lan-guage Pathology vol 17 no 1 pp 13ndash25 2015

[45] M Stella and Y N Kenett ldquoViability in multiplex lexicalnetworks and machine learning characterizes human crea-tivityrdquo Big Data and Cognitive Computing vol 3 no 3 p 452019

[46] M Stella ldquoModelling early word acquisition through multi-plex lexical networks and machine learningrdquo Big Data andCognitive Computing vol 3 no 1 p 10 2019

[47] S Beliga A Mestrovic and S Martincic-Ipsic ldquoSelectivity-based keyword extraction methodrdquo International Journal onSemantic Web and Information Systems vol 12 pp 1ndash262016

[48] S Beliga A Mestrovic and S Martincic-Ipsic ldquoToward se-lectivity based keyword extraction for Croatian newsrdquo CEURWorkshop Proceedings vol 1310 2014

[49] I T Koponen and M Nousiainen ldquoLexical networks andlexicon profiles in didactical texts for science educationrdquo inProceedings of the International Conference on ComplexNetworks and Ceir Applications pp 15ndash27 Springer LisbonPortugal December 2019

[50] C S Siew ldquoUsing network science to analyze concept maps ofpsychology undergraduatesrdquo Applied Cognitive Psychologyvol 33 no 4 pp 662ndash668 2019

[51] E Yli-Panula A Virta and K Merenluoto ldquoA Graph-the-oretic perspective on the content structure of physics lessonsand its relation to Student learning gainsrdquo in LearningTeaching and Growth into Teacherhood in the Light of Subject-Didactical Research pp 55ndash71 University of Turku TurkuFinland 2011

[52] G Scardoni and C Laudanna ldquoCentralities based analysis ofcomplex networksrdquo in New Frontiers in Graph CeoryY Zhang Ed pp 323ndash348 Intech Open London UK 2012

[53] K M Carley ldquoNetwork Text Analysis the network position ofconceptsrdquo in Text Analysis for the Social SciencesC W Roberts Ed pp 79ndash102 Lawrence ErlbaumAssociatesMahwah NJ USA 1997

[54] A E Motter A P S De Moura Y C Lai and P DasguptaldquoTopology of the conceptual network of languagerdquo PhysicalReview EmdashStatistical Physics Plasmas Fluids and RelatedInterdisciplinary Topics vol 65 no 6 2002

[55] G Bianconi S Dorogovtsev and J Mendes ldquoMutuallyconnected component of network of networksrdquo PhysicalReview E vol 91 no 1 Article ID 012804 2015

[56] E Estrada and J Gomez-Gardentildees ldquoCommunicability revealsa transition to coordinated behavior in multiplex networksrdquoPhysical Review E vol 89 no 4 Article ID 042819 2014

[57] G Menichetti D Remondini P Panzarasa R Mondragonand G Bianconi ldquoWeighted multiplex networksrdquo PLoS Onevol 9 no 6 Article ID e97857 2014

[58] J Gao S V Buldyrev H E Stanley and S Havlin ldquoNetworksformed from interdependent networksrdquo Nature Physicsvol 8 no 1 pp 40ndash48 2012

[59] R E Krathwohl ldquoA revision of bloomrsquos taxonomy anoverviewrdquo Ceory Into Practice vol 41 no 4 pp 213ndash2182002

16 Complexity

[60] M Bastian S Heymann and M Jacomy ldquoGephi an opensource software for exploring and manipulating networksrdquo inProceedings of the International AAAI Conference on Weblogsand Social Media San Jose CA USA 2009

[61] V Nicosia and V Latora ldquoMeasuring and modellling cor-relations in multiplex networksrdquo Physical Review E Statis-tical Nonlinear and Soft Matter Physics vol 92 no 3 ArticleID 032805 2015

[62] V D Blondel J-L Guillaume R Lambiotte and E LefebvreldquoFast unfolding of communities in large networksrdquo Journal ofStatistical Mechanics Ceory and Experiment vol 2008no 10 Article ID P10008 2008

[63] M De Domenico V Nicosia A Arenas and V LatoraldquoStructural reducibility of multilayer networksrdquo NatureCommunications vol 6 no 1 p 6864 2015

[64] H F de Arruda F N Silva L d F Costa and D R AmancioldquoKnowledge acquisition a complex networks approachrdquoInformation Sciences vol 421 pp 154ndash166 2017

[65] R Barnett Realizing the University in an Age of Super-complexity Vol 40 Society for Research into Higher Edu-cation amp Open University Press Philadelphia PA USA 2000

Complexity 17

[60] M Bastian S Heymann and M Jacomy ldquoGephi an opensource software for exploring and manipulating networksrdquo inProceedings of the International AAAI Conference on Weblogsand Social Media San Jose CA USA 2009

[61] V Nicosia and V Latora ldquoMeasuring and modellling cor-relations in multiplex networksrdquo Physical Review E Statis-tical Nonlinear and Soft Matter Physics vol 92 no 3 ArticleID 032805 2015

[62] V D Blondel J-L Guillaume R Lambiotte and E LefebvreldquoFast unfolding of communities in large networksrdquo Journal ofStatistical Mechanics Ceory and Experiment vol 2008no 10 Article ID P10008 2008

[63] M De Domenico V Nicosia A Arenas and V LatoraldquoStructural reducibility of multilayer networksrdquo NatureCommunications vol 6 no 1 p 6864 2015

[64] H F de Arruda F N Silva L d F Costa and D R AmancioldquoKnowledge acquisition a complex networks approachrdquoInformation Sciences vol 421 pp 154ndash166 2017

[65] R Barnett Realizing the University in an Age of Super-complexity Vol 40 Society for Research into Higher Edu-cation amp Open University Press Philadelphia PA USA 2000

Complexity 17