my book new

1

Department for Computer Science & Management

DECISION SUPPORT SYSTEMS

An Overview of Managerial Decision Support, Business Intelligence and Analytics

Marek Lubicz

www.ioz.pwr.wroc.pl/pracownicy/lubiczContact: B4 509, Wed 13-15, Thu 11-13

[email protected]

Portions

Marek Lubicz DSS 2012

What’s a DSS? Why DSS?

2

Plan of the Book

• Part I - Decision Making and Computerized Support1. Management Support Systems: An Overview2. Decision Making, Systems, Modeling, and Support

• Part II - Decision Support Systems3. Decision Support Systems: An Overview4. Modeling and Analysis5. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analysis, and Visualization6. Decision Support System Development

• Part III - Collaboration, Communication, Enterprise Decision Support Systems, and Knowledge Management

7. Collaborative Computing Technologies: Group Support Systems8. Enterprise Information Systems9. Knowledge Management

• Part IV – Intelligent Decision Support Systems10. Artificial Intelligence and Expert Systems: Knowledge-Based System11. Knowledge Acquisition, Representation, and Reasoning12. Advanced Intelligent Systems13. Intelligent Systems Over the Internet

• Part V – Implementing MSS in the E-Business Era14. Electronic Commerce15. Integration, Impacts, and the Future of the Management-Support Systems

Copyright © 2014 Pearson Education, Inc.

Decision Support Systems and Intelligent Systems, 7/ETurban, Aronson, LiangISBN-10: 0130461067 ©2005 • Prentice Hall

Plan of the Book

• Part I - Decision Support and Business Intelligence 1. Decision Support Systems and Business Intelligence

• Part II - Computerized Decision Support 2. Decision Making, Systems, Modeling, and Support3. Decision Support Systems Concepts, Methodologies, and Technologies: An Overview4. Modeling and Analysis

• Part III - Business Intelligence 5. Data Mining for Business Intelligence6. Artificial Neural Networks for Data Mining7. Text and Web Mining8. Data Warehousing9. Business Performance Management

• Part IV - Collaboration, Communication, Group Support Systems, and Knowledge Management

10. Collaborative Computer-Supported Technologies and Group Support Systems11. Knowledge Management

• Part V - Intelligent Systems12. Artificial Intelligence and Expert Systems13. Advanced Intelligent Systems

• Part VI - Implementing Decision Support Systems and Business Intelligence14. Management Support Systems: Emerging Trends and Impacts


Decision Support and Business Intelligence Systems, 9/ETurban, Sharda, DelenISBN-10: 013610729X ©2011 • Prentice Hall

3

Plan of the Book

• Chapter 1 - An Overview of Business Intelligence, Analytics and Decision Support

• Chapter 2 – Data Warehousing• Chapter 3 - Business Reporting, Visual Analytics &

Business Performance Management• Chapter 4 – Data Mining• Chapter 5 – Text, Web, and Social Analytics• Chapter 6 – Big Data and Analytics• Chapter 7 – Business Analytics: Emerging Trends

and Future Directions


Business Intelligence: A Managerial Perspective on Analytics, 3/ESharda, Delen, TurbanISBN-10: 0133051056©2014 • Prentice Hall

Plan of the Book

• Part I - Decision Making and Analytics: An Overview 1. An Overview of Business Intelligence, Analytics, and Decision Support2. Foundations and Technologies for Decision Making

• Part II - Descriptive Analytics 3. Data Warehousing4. Business Reporting, Visual Analytics, and Business Performance Management

• Part III - Predictive Analytics 5. Data Mining6. Techniques for Predictive Modeling7. Text Analytics, Text Mining, and Sentiment Analysis8. Web Analytics, Web Mining, and Social Analytics

• Part IV - Prescriptive Analytics9. Model-Based Decision Making: Optimization and Multi-Criteria Systems10. Modeling and Analysis: Heuristic Search Methods and Simulation11. Automated Decision Systems and Expert Systems12. Knowledge Management and Collaborative Systems

• Part V - Big Data and Future Directions for Business Analytics13. Big Data and Analytics14. Business Analytics: Emerging Trends and Future Impacts


Business Intelligence and Analytics: Systems for Decision Support, 10/ESharda, Delen, Turban, Aronson & Liang ISBN-10: 0133050904 • ISBN-13: 9780133050905 ©2015 • Prentice Hall

4

Decision Support Systems and Intelligent Systems2005PART I: Decision Making and Computerized Decision Support1. Management Support Systems: An Overview2. Decision-Making Systems, Models, and Support

PART II: Decision Support Systems3. Decision Support Systems: An Overview4. Modeling and Analysis5. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization6. Decision Support System Development

PART III: Collaboration, Communication, Enterprise Decision Support, and Knowledge Management7. Collaborative Computing Technologies: Group Support Systems8. Enterprise Information Systems9. Knowledge ManagementPART IV: Intelligent Decision Support Systems10. Intelligent Decision Support Systems11. Knowledge Acquisition, Representation, and Reasoning12. Advanced Intelligent Systems13. Intelligent Systems Over the InternetPART V: Implementing MSS in the E-Business Era14. Electronic Commerce15. Integration, Impacts, and the Future of Management-Support Systems

Decision Support and Business IntelligenceSystems 2011PART I: Decision Support and Business Intelligence1: Decision Support Systems and Business IntelligencePART II: Computerized Decision Support2: Decision Making, Systems, Modeling, and Support3: Decision Support Systems Concepts, Methodologies, and Technologies: An Overview4: Modeling and AnalysisPART III: Business Intelligence5: Data Mining for Business Intelligence6: Artificial Neural Networks for Data Mining7: Text and Web Mining8: Data Warehousing9: Business Performance ManagementPART IV: Collaboration, Communication, Group Support Systems, and Knowledge Management10: Collaborative Computer-Supported Technologies and Group Support Systems11: Knowledge ManagementPART V: Intelligent Systems12: Artificial Intelligence and Expert Systems13: Advanced Intelligent SystemsPART VI: Implementing Decision Support Systems and Business Intelligence14: Management Support Systems: Emerging Trends and Impacts

Business Intelligence and Analytics. Systems for Decision Support 2015PART I: Decision Making and Analytics: An Overview

PART II: Descriptive Analytics

PART III: Predictive Analytics

PART IV: Prescriptive Analytics

PART V: Decision Making and Analytics: An Overview

PART VI: Big Data and Future Directions for Business Analytics

Business Intelligence. A ManagerialPerspective on Analytics. 20141. An Overview of Business Intelligence, Analytics and Decision Support2. Data Warehousing3. Business Reporting, Visual Analytics & Business Performance Management4. Data Mining5. Text, Web, and Social Analytics6. Big Data and Analytics7. Business Analytics: Emerging Trends and Future Directions

DSS – BI – BA - MSS: terminology

Intelligent decision support systems

[Clark et al., MIS Quarterly, v. 31, 2007]Management Support Systems (MSS)

There have been calls for a new theory of management decision support that focuses on a broader context than does the traditional DSS to include business processes, organizational members, technology, infrastructure, and organizational outcomes from using the systems.

The field tends to continually chase the buzz words and system types of the day, often at the expense of establishing something of greater value that transcends system type and that provides a stronger foundation for the field.

5

Changing Business Environment & Computerized Decision Support

• Companies are moving aggressively to computerized support of their operations Business Intelligence• Business Pressures–Responses–Support Model

– Business pressures result of today's competitive business climate– Responses to counter the pressures – Support to better facilitate the process


The Business Environment

• The environment in which organizations operate today is becoming more and more complex, creating opportunities, and problems.– Example: globalization.

• Business environment factors: – markets, consumer demands, technology, and societal…

FACTOR DESCRIPTIONMarkets Strong competition

Expanding global marketsBlooming electronic markets on the InternetInnovative marketing methodsOpportunities for outsourcing with IT supportNeed for real-time, on-demand transactions

Consumer Desire for customizationdemand Desire for quality, diversity of products, and speed of delivery

Customers getting powerful and less loyalTechnology More innovations, new products, and new services

Increasing obsolescence rateIncreasing information overloadSocial networking, Web 2.0 and beyond

Societal Growing government regulations and deregulationWorkforce more diversified, older, and composed of more womenPrime concerns of homeland security and terrorist attacksNecessity of Sarbanes-Oxley Act and other reporting-related legislationIncreasing social responsibility of companiesGreater emphasis on sustainability


6

Computerized Decision Support

Organizational Responses: Managers may take actions, such as– Employ strategic planning– Use new and innovative business models– Restructure business processes– Participate in business alliances– Improve corporate information systems– Improve partnership relationships– Encourage innovation and creativity– Improve customer service and relationships– Move to electronic commerce (e-commerce)– Use new IT to improve communication, data access (discovery of information), and collaboration– Respond quickly to competitors' actions (e.g., in pricing, promotions, new products and services)– Automate many tasks of white-collar employees– Automate certain decision processes– Improve decision making by employing analytics

One of the major objectives of computerized decision support is to facilitate closing the gap between the current performance of an organization and its desired performance, as expressed in its mission, objectives, and goals, and the strategy to achieve them.

Computerized DSS can facilitate decision via:• Group communication and collaboration• Improved data management• Managing data warehouses and Big Data• Analytical support• Overcoming cognitive limits in processing and storing information• Knowledge management• Anywhere, anytime support

Managerial Decision Making

• Management is a process by which organizational goals are achieved by using resources. – Inputs: resources– Output: attainment of goals – Measure of success: outputs / inputs

• Management Decision Making• Decision making: selecting the best solution from two or more alternatives - alternative courses of action

for the purpose of attaining a goal or goals• Decision: conscious, free and non-random selection of one of the preset options, preceded by decision

analysis, i.e. considering the available options for action• Decision analysis: in managerial issues is based mostly on economic data, while options are typically

diverse management decisions: operational, tactical and strategic

• Decision making is difficult, because– Technology, information systems, advanced search engines, and globalization result in more and more

alternatives from which to choose– Government regulations and the need for compliance, political instability and terrorism, competition,

and changing consumer demands produce more uncertainty, making it more difficult to predict consequences and the future

– Other factors are the need to make rapid decisions, the frequent and unpredictable changes that make trial-and-error learning difficult, and the potential costs of making mistakes

7

examples of "difficult" decision-making situations [Bielecki]

- What are the optimal variants of production plans with different sets of resources?- What is the impact of various factors on changes in the option plan? What is a "probability" of each factor?- What will the future conditions of operation (e.g. interest rates, inflation, unemployment, Euro exchange rate)?- What should be the criteria for evaluation of options? Which criteria are most important and to what extent?- What expenses are associated with various options for development?- Are there any restrictions on the investment? (e.g. what is the minimum rate of return? What is the limit for the budget?)- What solutions are preferred by competitors? What strategies, scenarios can be selected by the competitors and with what "probability"?- Do we have an impact on competitors?- What problems or opportunities may arise in the near future, which currently cannot be seen? What is their "likelihood" and what does it depend on?

Managerial Decision Making

Decision-Making Process

Managers usually make decisions by following a four-step process (a.k.a. the scientific approach) 1. Define the problem (or opportunity) 2. Construct a model that describes the real-world problem.3. Identify possible solutions to the modeled problem and evaluate the solutions.4. Compare, choose, and recommend a potential solution to the problem.

Simon’s Decision-Making Process


8

[Griffin] CLASSICAL MODEL OF DECISION-MAKINGNormative approach to decision making (hypothetical); assumes:• logical and rational nature of managers• decisions are always in the interest of the organization• we make the decision using complete and perfect (precise, error-free) information on the decision

situation and possible options• decision-maker is able to successfully remove the uncertainty• decision-maker is able to rationally and logically assess all aspects of the decision situationincludes:• understanding (finding existence) and define the decision situation (premises)• identifying options for action• assessment of each option in terms of: feasibility, adequacy and consistency• choosing the best variant (in specific circumstances)• implement the selected option• observation and evaluation of the implementationThis approach is also called the scientific method of problem solving [Ackoff]

[Herbert Simon, according to Griffin] ADMINISTRATIVE MODEL OF DECISION-MAKING: incorporates the derogations from the classic model in real-life

decision-making processes; assumptions:• decisions are not always taken in accordance with the rules of logic and rationality - there are restrictions

on rationality• decision-makers are forced to use incomplete and imperfect (inaccurate, erroneous) information on the

decision situation and the possible variants• decision-maker is willing (or is forced) to accept the first feasible solution found• as a result decision-makers decide (as they have to), but not necessarily they select the option that

actually best serves the interest of the organization (not a "best possible")

Decision-Making Process

An Early Decision Support Framework (by Gory and Scott-Morten, 1971)


• Degree of Structuredness (Simon, 1977)– Decisions are classified as

• Highly structured (a.k.a. programmed)• Semi-structured• Highly unstructured (i.e., nonprogrammed)

• Types of Control (Anthony, 1965)– Strategic planning (top-level, long-range)– Management control (tactical planning)– Operational control

9

DECISION CATEGORIES BY DEGREE OF PROBLEM STRUCTUREDNESS• Structured: situations where the procedures to follow when a decision is needed can be specified in

advance: Repetitive, Standard solution methods exist, Complete automation may be feasible• Unstructured: decision situations where it is not possible to specify in advance most of the decision

procedures to follow: One-time, No standard solutions, Rely on judgment, Automation is usually infeasible• Semi-structured: decision procedures that can be pre-specified, but not enough to lead to a definite

recommended decision: Some elements and/or phases of decision making process have repetitive elements

DSSs most useful for repetitive aspects of semi-structured problems

- structured problems: [development of LC Towers in Wrocław]- purpose of the activities and the method of achieving the goals (set of solutions) are defined- all relevant parameters and decision variables are quantifiable and known- all activities to perform can be uniquely represented in the form of an algorithm- can be complex and tedious to solve

- unstructured problems: [improvement of public transport in Wrocław]:- existence of many decision makers / stakeholders, each of which sees the problem differently- existence of multiple criteria, which are generally unknown in advance, arising during evaluation of the

solutions- considerable degree of uncertainty regarding many aspects of the problem (not everything can be

expressed numerically)- existence of a conglomerate of problems, which should be considered to enable understanding the main

decision problem (or: which should be solved before one can start solving the main decision problem)- it is not clear how to define a set of solutions- it is not clear how to define a set of activities that may lead to the implementation of solutions once they

are found- difficult to define the decision-making procedures (fuzzy procedures, limited extent of algorithmization)

DSS for Management Support: support what - problems

Computer Support for Structured Decisions

• Structured problems: encountered repeatedly, have a high level of structure• It is possible to abstract, analyze, and classify them into specific categories

(*) e.g., make-or-buy decisions, capital budgeting, resource allocation, distribution, procurement, and inventory control

• For each category a solution approach is developed => Management Science (also referred to as OperationsResearch)

In solving structured problems, managers should follow the five-step MS approach• Define the problem • Classify the problem into a standard category (*)• Construct a model that describes the real-world problem• Identify possible solutions to the modeled problem and evaluate the solutions• Compare, choose, and recommend a potential solution to the problem

10

Automated Decision Making

• A relatively new approach to supporting decision making• Applies to highly structured decisions• Automated decision systems (ADS)

(or decision automation systems) • An ADS is a rule-based system that provides a solution to a repetitive managerial problem in a specific

area – e.g., simple-loan approval system

• ADS initially appeared in the airline industry called revenue (or yield) management (or revenue optimization) systems

– dynamically price tickets based on actual demand• Today, many service industries use similar pricing models• ADS are driven by business rules!


Computer Support for Unstructured Decisions

• Unstructured problems can be only partially supported by standard computerized quantitative methods

• They often require customized solutions• They benefit from data and information• Intuition and judgment may play a role• Computerized communication and collaboration technologies along

with knowledge management is often used

• Solving semi-structured problems may involve a combination of standard solution procedures and human judgment

• MS handles the structured parts while DSS deals with the unstructured parts

• With proper data and information, a range of alternative solutions, along with their potential impacts

11

DECISION CATEGORIES BY DEGREE OF PROBLEM STRUCTUREDNESS

WELL-STRUCTURED PROBLEMS ILL-STRUCTURED PROBLEMS

Stable decision situation (premises) Unstable or unexpected decision situation

Routine decisions Decisions that require creativity and reflections

Decision-making situation fully understood and described (all data available)

Decision-making situation uncertain, incomplete information

Decisions repeatable One-time or unique decisions

Specialized, domain specific General, multidisciplinary

Self-evident decision maker and stakeholders

(entities or individuals such as an employee, customer or citizen, who are involved with an organization, society, etc. and therefore has responsibilities towards it and an interest in its success)

Unclear decision maker and stakeholders

(who, if anybody, is „responsible” for or should react to increasing mortality due to lung cancer or environment pollution ?)


Well-structured Ill-structured SimonHard problems Soft problems

Messy problems Ackoff

Tame problems (trivial) Wicked problem (deliberate) Rittel/ WeberSwamp (awkward, cumbersome) Schon

Concept of Decision Support Systems

Classical Definitions of DSS

Interactive computer-based systems, which help decision makers utilize data and models to solve unstructured problems„ - Gorry and Scott-Morton, 1971

Decision support systems couple the intellectual resources of individuals with the capabilities of the computer to improve the quality of decisions. It is a computer-based support system for management decision makers who deal with semi-structured problems - Keen and Scott-Morton, 1978.......................................................................................................................................................................................................................

DSS as an Umbrella Term / as a Methodology

[Turban 2005] A Decision Support System - methodology that supports decision-makingManagement Support Systems - The support of management tasks by the application of technologies; sometimes called Decision Support Systems or Business Intelligence

[Alter 2004] Decision support is the use of any plausible computerized or non-computerized means for improving sense making and/or decision making in a particular repetitive or non-repetitive business situation in a particular organization. Decision support is not about tools per se, but rather, about making better decisions within work systems in organizations. .......................................................................................................................................................................................................................

DSS as a Specific Application / as a System[Marakas 2003] A DSS is a system under the control of one or more decision makers that assists in the process of decision making by providing an organized set of tools to impart structure to portions of the decision-making situation and improve the ultimate effectiveness of the decision outcome

[Laskey 2006] A decision support system is a computer-based system that supports the decision making process, and has the following features:• Assist decision makers in semi-structured tasks • Support not replace human judgment• Highly interactive • Improve effectiveness of human decision makers

[Power, D. J., Free Decision Support Systems Glossary DSSResources.COM/glossary/]DSSs are interactive computer-based systems, intended to help decision makers use communications technologies, data, documents, knowledge and/or models to identify and solve problems, complete decision process tasks and make decisions

12

DSS as a Specific Application

• In a narrow sense DSS refers to a process for building customized applications for unstructured or semi-structured problems

• Components of the DSS Architecture– Data, Model, Knowledge/Intelligence, User, Interface (API and/or user

interface)– DSS often is created by putting together loosely coupled instances of these

components


DSS as an Umbrella Term / an Approach or Methodology

• The term DSS can be used as an umbrella term to describe any computerized system that supports decision making in an organization – E.g., an organization wide knowledge management system; a decision

support system specific to an organizational function (marketing, finance, accounting, manufacturing, planning, SCM, etc.)

Decision Support Systems 38 (2004) 319–327 A work system view of DSS in its fourth decade Steven Alter

The initial concept of DSS focused on using interactive computing in semi-structured decision making.The emphasis on semi-structured decision making seemed important (in academic politics if not in other ways) because that distinguished DSS from OR, especially from optimization models, which attempted to automate decision making, or so it seemed.After 30+ years, the original issues that led to the DSS movement have receded to ancient history. Computers are used interactively by managers, non-managers, and school children. Computerized data and models are used so commonly and for so many structured, semi-structured, and unstructured tasks that the non-use of computers in typical decision-oriented situations is sometimes a noteworthy exception. With today’s widespread adoption of PCs and the Internet, we should simply declare victory on the original DSS agenda that included interactive computing, application of computing to semi-structured problems, use of computers by managers, and the ability to analyze data and models.…. little can be said about DSS in general other than statements such as ‘‘Systems of types X, Y, and Z are typically included under the general umbrella of DSS.’’ By placing disparate approaches under the same umbrella, the broader, more encompassing definitions of DSS tend to blur any distinguishing characteristics. In effect, DSS becomes all information systems that are used by managers or business professionals and do not fall into some other category.DSS has little meaning other than as an umbrella covering a cluster of research interests related to using technology to support sense making and decision making.

13

contemporary meaning of ”DSS” -> INFORMS

The emphasis on semi-structured decision making seemed important (in academic politics if not in other ways) because that distinguished DSS from OR, especially from optimization models, which attempted to automate decision making, or so it seemed.

contemporary meaning of ”DSS” -> INFORMS

[Power, D. J., Free Decision Support Systems Glossary DSSResources.COM/glossary/]DSSs are interactive computer-based systems, intended to help decision makers use communications technologies, data, documents, knowledge and/or models to identify and solve problems, complete decision process tasks and make decisions

The Institute for Operations Research and the Management Sciences (INFORMS) is the largest society in the world for professionals in the field of operations research (O.R.), management science, and analytics.

14

DECISION CATEGORIES BY REPEATABILITY

[Griffin]: programmed decisions (to complete the structure and / or repeated with some frequency)and not programmed (not very clear structure, made much less than the programmed decisions)

Inaccurate definition: not so much the frequency is important to take, but the natureof the decision problem, indeed elsewhere Griffin writes about the characteristics of such problems: a unique, absorbing much time and resources necessary for a thorough examination of decision situation.The main factors in such decisions are intuition and experience.

Examples: most of the decisions made by top managers and policy makers, strategic planning, designing the structure of: an organization, new plants / products, legal issues, contracts.


DECISION CATEGORIES BY NATURE OF PROBLEMS(also called a taxonomy by mode of assistance)

- Issues related to the processing of large databases

- Problems in which complex calculations using knownnumerical algorithms are required

- Those in which the "problem" is lack of precision (in theavailable data, the absence of rules of inference, datainterpretation, in the preparation of a potential user) -common problems for an "expert"

- Those in which the most important is the flexibility andease of use of data from various sources (including, forexample - different environments), and propercommunication of the results of inference

PROBLEM CATEGORIES

[„database”]

[„optimisation”]

[„expert knowledge”]

[„integration and communication”]


15


Intelligence Scan the environment Analyze organizational goalsCollect dataIdentify problemCategorize problem [Structured, Unstructured; Decomposed into smaller parts]Assess ownership and responsibility for problem resolution

Design Develop alternative courses of actionAnalyze potential solutionsCreate modelTest for feasibility Validate resultsSelect a principle of choice [Establish objectives; Incorporate into models; Risk assessment and acceptance; Criteria and constraints]

Choice Principle of choice [Describes acceptability of a solution approach]Normative Models; Descriptive Models[Optimization; Rationalization; Suboptimization]Decision making with commitment to actDetermine courses of action [Analytical techniques; Algorithms; Heuristics; Searches]Analyze for robustness

Implementation

Putting solution to workVague boundaries [Dealing with resistance to change; User training; Upper management support]


DECISION CATEGORIES BY INFORMATION BACKGROUND AVAILABLE - scope

DSS for Management Support: based on the - data


16

DECISION CATEGORIES BY INFORMATION BACKGROUND AVAILABLE - scope

decision-makers take into account various assumptions, which - depending on the degree of aggregation logical and information contents - may be called:

- facts: single numbers, graphics, verbal descriptions[e.g. unemployment rate in Lower Silesia in 2009]

- data: collection of facts, for a fixed aspect of reality [e.g. dynamics of unemployment rate in Lower Silesia 1999-2009]

- information: sets of data, presented in a meaningful way (related to a particular decision making process), data that have been categorized and classified, or otherwise ordered, a description of states of affairs, events, processes [e.g. changes in unemployment rates by age group, sex, education]

- knowledge: (a) ordered and "cleaned" information (quantitative, factual or encyclopedicknowledge); (b) interpreted information, indicating the importance of contextual and logical relationships of cause and effect relationships ("dependencies, trends, patterns and the law"), determined subjectively perceived by the decision maker; arises when drawing conclusions from the available data and information (qualitative knowledge)

- or wisdom: ability to use knowledge and experience to make good decisions and judgments


DECISION CATEGORIES BY INFORMATION BACKGROUND AVAILABLE – completeness and certainty

- Decision making under certainty: when the decision maker knows with reasonable certainty the scope of available options and associated with each of these conditions- Decision making under risk: the availability of each possibility and its potential benefits and costs are known with some estimated "probability"- Decision making under uncertainty: decision-maker does not know all the opportunities, risks associated with each of them and / or their likely consequences

most important decisions in contemporary organizations are decisions under uncertainty, due to the complexity of the organization and dynamic features such as the organizations themselves and their environmentconsequently to make effective decisions it is important to gain as much information as available, and to perform reasoning in a logical and rational way, but an equally important role is played by intuition, judgment and experience of the decision-maker


Documents

my book new