Multiplication of two 3 d sparse matrices using 1d arrays and linked lists

International Journal of Modern Electronics and Communication Engineering (IJMECE) ISSN: 2321-2152 (Online) Volume No.-1, Issue No.-2, May, 2013

RES Publication©2012 Page | 10

http://www.resindia.org

New Algorithms for Multiplication of Two 3D Sparse

Matrices Using 1D Arrays and Linked Lists

Abhishek Jain 1st

Department of Computer Engineering,

Poornima College of Engineering,

Jaipur, India

E-mail: [email protected]

Sandeep Kumar 2nd

Faculty of Engineering & Technology,

Jagan Nath University,

Jaipur, India

E-mail: [email protected]

Abstract: A basic algorithm of 3D sparse matrix multiplication (BASMM) is presented using one dimensional (1D) arrays which is used further

for multiplying two 3D sparse matrices using Linked Lists. In this algorithm, a general concept is derived in which we enter non- zeros elements

in 1st and 2nd sparse matrices (3D) but store that values in 1D arrays and linked lists so that zeros could be removed or ignored to store in

memory. The positions of that non-zero value are also stored in memory like row and column position. In this way space complexity is

decreased. There are two ways to store the sparse matrix in memory. First is row major order and another is column major order. But, in this

algorithm, row major order is used. Now multiplying those two matrices with the help of BASMM algorithm, time complexity also decreased.

For the implementation of this, simple c programming and concepts of data structures are used which are very easy to understand for everyone.

Keywords: sparse, non-zero values, dimension, depth, limit

I. INTRODUCTION

A matrix, in which maximum elements of matrix are zero, is

called sparse matrix. A matrix that is not sparse is called dense

matrix. [3]

Consider the sparse matrix given below:-

The natural method of sparse matrices in memory as two

dimensional arrays may not be suitable for sparse matrices

because it will waste our memory if we store „zero‟ in a block

by 2 bytes. So, we should store only „non-zero‟ elements. [3]

One of the basic methods for storing such a sparse matrix is to

store non-zero elements in a one-dimensional array and to

identify each array element with row and column indices as

shown in figure below.

There are two ways to represent the sparse matrix

1.1. Using array

1.2. Using linked list

In both representations, information about the non-zero

element is stored.

1.1. Using Array:-

Below figure shows representation of sparse matrix as an

array.

Note that the sparse matrix elements are stored in row major

order with zero elements removed. Any matrix, with larger

quantity of zeros then non-zeros is said to be sparse matrix.

1.2. Using Linked List:-

A sparse matrix is given below:-

Representation of sparse matrix in c is given below:-

struct matrix

{

int value, row, column;

struct matrix * next;

} *start;

The sparse matrix pictorial represented as:-[3]




Phases

The whole work is divided in two phases which are given

below:-

i. Multiplication of two 3D sparse matrices using 1D

array

ii. Multiplication of two 3D sparse matrices using linked

list

II. PHASE 1. PROPOSED ALGORITHM FOR

MULTIPLICATION OF TWO 3D SPARSE MATRICES

USING 1D ARRAY

If a matrix is a size of m x n x d, then here m is for number of

rows, n is for number of columns and d is for number of faces

or depth. Here take m=n=d=3.

Let us understand how non-zero values are stored in sparse

matrix. We use three arrays, one for row position, second for

column position and third for storing non-zero values for each

matrix. In a sparse matrix has size of m x n, then the maximum

number of non-zero values is calculated by limit = (d*m*n)/2.

Consider three matrices A, B and C. For A matrix to storing

row positions array is denoted by ar, to storing column

positions array is denoted by ac and to non-zero values array is

denoted by av. Similar for B and C Matrices.

Now, Firstly we take first non-zero value from A matrix and

note its position. Now we compare its column position with

row position of B matrix. If it matches, we multiply A matrix‟s

non-zero value with B matrix‟s non-zero value and store the

result in value array of C matrix, and store row position from

A matrix and column position from B matrix to row and

column position of C matrix respectively.

If it not matches, then we compare A‟s column position to the

next B‟s row position in array. This process repeated until all

elements of A matrix multiplied with B matrix. In this process,

we get some non-zero values of same positions, so we have to

add them to get final result.

limit1=maximum non-zero values in 1st sparse matrix (d1 x r1

x c1)

limit2=maximum non-zero values in 2nd sparse matrix (d2 x

r2 x c2)

k1=non-zero values in 1st matrix

k2 =non-zero values in 2nd matrix

k3= pointer for 3rd matrix of multiplication

t1=traversing pointer for 1st matrix

t2= traversing pointer for 2nd matrix

ad[limit1],ar[limit1], ac[limit1], av[limit1] – arrays of 1st

matrix for storing positions of depth,rows and columns and for

values.

bd[limit2],br[limit2], bc[limit2], bv[limit2] – arrays of 2nd

matrix for storing positions of depth, rows & columns and for

values.

cd[limit1*clo2],cr[limit1*col2], cc[limit1*col2],

cv[limit1*col2] – arrays of 3rd matrix for storing positions of

depth, rows & columns and for values after multiplication.

1. t1=0,t2=0;

2. while(t1<k1)

3. {

4. match=0;

5. while(t2<k2)

6. {

7. count=0;

8. if(br[t2]==ac[t1] && ad[t1]== bd[t2])

9. {

10. while(count<c2)

11. {

12. if( bc[t2]==count)

13. {

14. int f=0;

15. for(int y=0;y<k3;y++)

16. {

17. if( cd[y]== ad[t1] && cr[y] == ar[t1] && cc[y]

== bc[t2])

18. {

19. f=1;//flag

20. cv[y] = cv[y] + av[t1] * bv[t2];




21. break;

22. }

23. }

24. if(f==0)

25. {

26. cd[k3] = ad[t1]; //or ad[t2] becoz both are same

27. cr[k3] = ar[t1];

28. cc[k3] = bc[t2] ;

29. cv[k3] = av[t1] * bv[t2];

30. k3++;

31. count++;

32. }//while

33. }//if

34. if(match==c2)

35. {

36. break;

37. }

38. t2++;

39. }//while

40. t2=0;

41. t1++;

42. }//while

43. }

44. }//if

III. PHASE 2. PROPOSED ALGORITHM FOR

MULTIPLICATION OF TWO 3D SPARSE MATRICES

USING LINKED LISTS

This is totally different concept from multiplication of 2D or

3D sparse matrix multiplication using 1D array. Here we use

linked lists instead of 1D arrays to store information of sparse

matrices.

In this new approach, the structure of liked list is modified and

created two new structures of linked list to store information

about sparse matrix.

First structure is node1, which hold three information, first-

row position of non-zero values, second-down pointer which

store the address of next node of non-zero value and third-

right pointer which store the address of node2. Start pointers

store the address of first node of each matrix. Like start1 store

the address of A matrix, start2 store the address of B matrix

and start3 store the address of resultant C matrix.

Now, second structure node2, which also hold three

information, first- col to store column position of non-zero

element, second- value which store non-zero value and third-

next which store the address of next node which hold the non-

zero values in same row if it has. In this way, sparse matrix

information is totally re-organized in better structure like

matrix form in which non-zero values of same row are linked

with each other in one line and next row is linked separately in

next line but lined with previous row by down pointer.

In this way, multiplication algorithm is also similar to

traditional multiplication approach but in traditional approach,

multiplication loop depends of matrix size and in this new

approach multiplication loop depends on number of non-zero

values.

This node3 used for third dimension (depth d or faces). It has 3

parts, first- depth which store the depth number of face

number of sparse matrix, second- start pointer which store the

address of either start1 or start2 or start3. It means, this start

pointer store the address of first node of first face of first

sparse matrix, similarly for all. Third- dnext which store the

address of next node of node3 structure who hold the similar

information of previous node. Like, next or second node of

node3 hold the information as in depth part it store second

face, in start part it store the address of first node of second

face of first sparse matrix. And dnext node holds the

information of third node of node3 structure.

Similarly all these information is also logically arranged for B

and resultant C matrix.

Information of each face will be multiplied with corresponding

same face and results will also store in same corresponding

face.

struct node1

{

int row;

struct node1 * down;

struct node2 * right;

} *start1, *start2, *start3;

struct node2

{

int col;

int value;

struct node2 * next;

};

struct node3 // singly linked list for depth node

{

int depth;

struct node3 * start;

struct node3 * dnext;

} * A_depth, * B_depth, * C_depth ;




Multiplication Algorithm for 3D Sparse Matrix using

Linked Lists.

1. Repeat step 2 until dnext pointer becomes NULL for

A matrix

2. Repeat step 3 and 4 until down pointer becomes

NULL for A matrix

3. If right pointer not equals to NULL for A matrix, then

4. Repeat step 5 until next pointer becomes NULL for A

matrix

5. Repeat step 6 until dnext pointer becomes NULL for

B matrix

6. Repeat step 7 and 8 until down pointer becomes

NULL for B matrix

7. If right pointer not equals to NULL for B matrix, then

8. Repeat step 9 until next pointer becomes NULL for B

matrix

9. If col part of A matrix is equal to row part of B

matrix, then

(a) Multiply element from A matrix‟s value part with

element from B matrix‟s value and store the result to

C matrix‟s value part.

(b) Copy row position from row part of A matrix to row

part of C matrix.

(c) Copy column position from column part of B matrix

to column part of C matrix.

10. Exit.

IV. ANALYSIS OF EXISTING AND PROPOSED

ALGORITHM IN TERMS OF TIME COMPLEXITY

For 3D sparse matrix, if

1st matrix = d * R1 x C1 = 3 x 3 x 3

2nd matrix =d * R2 x C2 = 3 x 3 x 3

limit1 = maximum no. of non-zero elements in 1st matrix

= (d x R1 x C1)/2 = 13

limit2 = maximum no. of non-zero elements in 2nd matrix

= (d x R2 x C2)/2 = 13

k1 = actual no. of non-zero elements in 1st matrix

k2 = actual no. of non-zero elements in 2nd matrix

E1 - Fast Transpose and Mmult Algorithm (Using Arrays)

(Existing Algorithm)

A1 - Proposed Algorithm for 3d Sparse Matrix

Multiplication Using 1D Arrays

A2 - Proposed Algorithm for 3d Sparse Matrix

Multiplication Using Linked Lists

Table 1.Ddifferent time complexities of given algorithms

Algorithm

Time

Complexity

formula

E1 О(d*R1*C1*C2)

A1 O(d*K1*K2*C2)

A2 O(d*K1*K2)




Table 2. Calculating time complexities of all three algorithms on different K

values

Table3. Comparison between existing and proposed 1st algorithm

Table4. Comparison between existing and proposed 2nd algorithm

Graph : Comprasion among all three algorithms

V. CONCLUSION AND FUTURE WORK

Normally space and time complexity is greater if we store non-

zero values in 1D array but lesser if we stored those values in

linked list.

Now, all time complexity depends on number of comparisons,

multiplication steps and number of non-zero values in sparse

matrix. If we decrease number of comparisons then we can

decrease time complexity. if 1st matrix is of d x R1 x C1 order,

K1 is the number of non-zero values in 1st matrix, 2

nd matrix is

of d x R2 x C2 order, K2 is the number of non-zero values in

2nd

matrix and d is the depth or dimension of sparse matrix,

then time complexity for A1 algorithm for 3D sparse matrix

using 1D arrays is d*K1*K2*C2 and for A2 algorithm for 3D

sparse matrix using linked lists is d*K1*K2.

At present, both algorithms are suitable only for multiplication

for two 3D sparse matrices. But in future it could be

generalized for multidimensional arrays.

REFERENCES

[1] Ellis Horowitz and Sartaz Sahni – “Fundamentals of

data structures” (2012), Galgotia Booksource (Page

51-61)

[2] Indra Natarajan – “Data structure using C++” (2012),

Galgotia Booksource. (Page 11-164, 297-346)

[3] Hariom Pancholi- “Data Structures and Algorithms

using C” (5th

ed. 2012), Genus Publication. ISBN

978-93-80311-01-2 (Page 2.26-2.30, 4.78)

[4] Cormen – “Introduction to Algorithms (2nd

ed.)”,

Prentice Hall of India. (Page 527-548). ISBN- 978-

81-203-2141-0

K1=K2=K Time Complexity

K E1 A1 A2

1 81 9 3

2 81 36 12

3 81 81 27

4 81 144 48

5 81 225 75

6 81 324 108

7 81 441 147

8 81 576 192

9 81 729 243

10 81 900 300

11 81 1089 363

12 81 1296 432

13 81 1521 507

Condition Case

Time Complexity

Comparison

E1 A1

d*K1*K2 <

d*R1*C2

Best

Case High Low

d*K1*K2 =

d*R1*C2

Average

Case High High

d*K1*K2 >

d*R1*C2 Worst Case Low High

Condition Case

Time Complexity

Comparison

E1 A2

d*K1*K2 <

d*R1*C2 Best Case High Low

d*K1*K2 =

d*R1*C2 Average Case High Low

d*K1*K2 >

d*R1*C2 and

K1 < √

(R1*C1*C2)

Below

Average Case Medium Low

d*K1*K2 >

d*R1*C2 Worst Case Low High




[5] R B Patel and M M S Rauthan – “Expert data

structure with C++ (2nd

ed.)”,ISBN-8187522-03-8,

(Page 262-264)

[6] Randolph E. Banky, Craig C. Douglasz – “Sparse

Matrix Multiplication Package (SMMP)” April 23,

2001. Advances in Computational Mathematics

[7] Golub, Gene H.; Van Loan, Charles F.(1996).-

“Matrix Computations” (3rd ed.). Baltimore: Johns

Hopkins. ISBN 978-0-8018-5414-9.

[8] Jess, J.A.G. –“ A Data Structure for Parallel L/U

Decomposition” Proc. IEEE Volume: C-31 , Issue: 3

Page(s): 231 – 239, ISSN : 0018-9340

[9] I. S. Duff - "A survey of sparse matrix

research", Proc. IEEE, vol. 65, pp. 500 -1977

[10] Stoer, Josef; Bulirsch, Roland (2002)-“Introduction to

Numerical Analysis” (3rd ed.). Berlin, New York:

Springer-Verlag. ISBN 978-0-387-95452-3.

[11] Pissanetzky, Sergio (1984)-“Sparse Matrix

Technology”, Academic Press.

[12] Reguly, I. ; Giles, M. – “Efficient sparse matrix-

vector multiplication on cache-based GPUs ” IEEE

Conference Publications, Year: 2012, Page(s): 1 – 12

[13] Daniel Krála, Pavel Neográdyb, Vladimir Kellöb-

“Simple sparse matrix multiplication algorithm”,

Computer Physics Communications, ELSEVIER,

Volume 85, Issue 2, February 1995, Pages 213–216,

[14] R. C. Agarwal ET AL-“A three dimensional approach

to parallel matrix multiplication”, IBM J. Res.

Develop. Vol. 39 No. 5 September 1995

AUTHOR’S BIOGRAPHIES

Abhishek Jain 1st

B.E. (CSE, SKIT College, Jaipur/UOR, 2009 Batch),

M.Tech (Pursuing) (CS, Jagan Nath University, Jaipur, 2011-13

Batch)

Sandeep Kumar 2nd

B.E. (CSE, ECK Kota/UOR, 2005 Batch)

M. Tech (ACEIT, Jaipur/RTU, 2011 Batch)

Ph. D. (Pursuing) (CSE, Jagan Nath University, Jaipur)

http://en.wikipedia.org/wiki/Springer-Verlag

http://en.wikipedia.org/wiki/International_Standard_Book_Number

http://en.wikipedia.org/wiki/Special:BookSources/978-0-387-95452-3

http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=p_Authors:.QT.Reguly,%20I..QT.&newsearch=true

http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=p_Authors:.QT.Giles,%20M..QT.&searchWithin=p_Author_Ids:38231352800&newsearch=true

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6339602&contentType=Conference+Publications&searchField%3DSearch_All%26queryText%3Dsparse+matrix

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6339602&contentType=Conference+Publications&searchField%3DSearch_All%26queryText%3Dsparse+matrix

http://www.sciencedirect.com/science/journal/00104655

http://www.sciencedirect.com/science/journal/00104655/85/2

Education

Multiplication of two 3 d sparse matrices using 1d arrays and linked lists