Pipe-It Manaul 20130806

8/11/2019 Pipe-It Manaul 20130806

1/264

PIPE-ITMANUAL

Copyright 2013 Petrostreamz AS.

All rights reserved.

2013-08-06

Questions may be addressed to:

[email protected]

Website:

http://petrostreamz.com
mailto:[email protected]:[email protected]://petrostreamz.com/http://petrostreamz.com/http://petrostreamz.com/mailto:[email protected]

8/11/2019 Pipe-It Manaul 20130806

2/264

8/11/2019 Pipe-It Manaul 20130806

3/264

Table of Contents

Pipe-It Advantages ..................................................................6

Getting Started .........................................................................8

Register for an account and a license ............................................................ 8Download Pipe-It Software ......................................................................... 10Pipe-It Installation ..................................................................................... 11Installed Components ................................................................................. 15Pipe-It Preferences..................................................................................... 15

How to use Pipe-It: A small working example ..................... 20

Pipe-It Example: Launch a GAWK program .................................................. 20

Help and Documentation ...................................................... 31

Tooltip ...................................................................................................... 31Pipe-It Help System ................................................................................... 31Support by email ........................................................................................ 32

The main software components ........................................... 33

Introduction............................................................................................... 33Terminology .............................................................................................. 35

Pipe-It Graphical Elements ................................................... 36

Resources .................................................................................................. 36Program (Script) ........................................................................................ 39Scripter Dialog Box ..................................................................................... 40Example .................................................................................................... 40Summary of Scripter Dialog Options ............................................................ 47

Pipe-It Projects & Sub-Projects ............................................ 51

Projects & Composites ................................................................................ 52Composites ................................................................................................ 53Export of Composite ................................................................................... 54Importing .................................................................................................. 55Sockets ..................................................................................................... 56Socket Types ............................................................................................. 57

Runner ...................................................................................................... 59

8/11/2019 Pipe-It Manaul 20130806

4/264

Using Projects by multiple users .................................................................. 59Example Pipe-It Projects ............................................................................. 60Designing Pipe-It projects ........................................................................... 62

Pipe-It Graphical User Interface .......................................... 65

Tool Bar .................................................................................................... 66

Multiple Views ....................................................................... 71

Operating Systems ..................................................................................... 72Interactions with Canvas ............................................................................ 73Save As Dialog ........................................................................................... 90Displaying the model status and results ....................................................... 91Exporting and Importing data ..................................................................... 92

Optimizer ............................................................................... 95

Introduction and example ........................................................................... 95Optimization .............................................................................................. 99The Solvers ............................................................................................... 99Solver plugin API ..................................................................................... 100Example .................................................................................................. 101Reference ................................................................................................ 102Solver Parameters .................................................................................... 114

Optimization Variables .............................................................................. 117

Random Sampler Solver ........................................................................... 118Case Matrix Solver ................................................................................... 118Nelder-Mead Simplex-Reflection Solver ...................................................... 118IPOPT Solver ........................................................................................... 119NOMAD ................................................................................................... 121The DotNumericsPlugin solvers ................................................................. 121

Linkz .................................................................................... 122

Introduction............................................................................................. 122Linkz Window .......................................................................................... 122

Creating a new link .................................................................................. 123Using Delimiters and Regular Expression ................................................... 126The Optimizer Connection ........................................................................ 128More than a single value .......................................................................... 128MapLinkz ................................................................................................. 130Links in Annotations ................................................................................. 131

Streamz ............................................................................... 132

Introduction............................................................................................. 132

8/11/2019 Pipe-It Manaul 20130806

5/264

Streamz Language Commands .................................................................. 134Streamz GUI (Streamz Assistants) ............................................................. 136Streamz Copier ........................................................................................ 145Streamz Tabulator.................................................................................... 146Streamz Separator ................................................................................... 149

Streamz Generic ...................................................................................... 150

Streamz Documentation ........................................................................... 150

Strexzel ............................................................................... 151

Strexzel Macro Commands ........................................................................ 151Comments ............................................................................................... 152

Accumulate.............................................................................................. 152Add Note ................................................................................................. 153Calculate ................................................................................................. 154

Change Stream Units................................................................................ 157

Edit Component ....................................................................................... 158Edit Variable ............................................................................................ 159Expose Components ................................................................................. 161Insert Component .................................................................................... 162Insert Streams ......................................................................................... 164Insert Variable ......................................................................................... 165Load Characterization ............................................................................... 168Move Component ..................................................................................... 170Move Streams .......................................................................................... 171

Move Variable .......................................................................................... 173Name Characterization ............................................................................. 174

Remove Columns ..................................................................................... 175Remove Component ................................................................................. 176Remove Streams ...................................................................................... 177Remove Variable ...................................................................................... 178Save Characterization ............................................................................... 179Set Constant ............................................................................................ 180Set Formula ............................................................................................. 181Set Precision ............................................................................................ 185

Seed ....................................................................................................... 187

Strexzel GUI ............................................................................................ 187Strexzel interaction with Pipe-It. ............................................................... 193Console mode / Command line arguments ................................................. 193

Convertors ........................................................................... 195

Ecl2Str .................................................................................................... 195Pre Ecl..................................................................................................... 197Sen2Str ................................................................................................... 202

Script Assistant ................................................................... 206

8/11/2019 Pipe-It Manaul 20130806

6/264

Overview ................................................................................................. 206Using Scripts............................................................................................ 206Creating Scripts ....................................................................................... 207Modifying Scripts...................................................................................... 209Workflow ................................................................................................. 210

Detailed Description ................................................................................. 211

Plotz..................................................................................... 219

Plotz GUI............................................................................................... 223Help ....................................................................................................... 224Graphical Interface and Functionality ........................................................ 228

Appendix ............................................................................. 232

Streamz Tabulation Reference .................................................................. 234

Terminology ............................................................................................ 237Advanced Linkz trick................................................................................. 242Interaction with 3rd party software ........................................................... 243Getting Started with Streamz (Application) ................................................ 245Plots on Canvas in Pipe-It projects and using "R" language ........................ 259Pipe-It File types ...................................................................................... 262Copyright Notice ...................................................................................... 263Disclaimer ............................................................................................... 263

8/11/2019 Pipe-It Manaul 20130806

7/264

Pipe-It Manual Page 6

Pipe-It AdvantagesPetrostreamzs Pipe-It is a new methodology for asset optimizationand exploitation where petroleum streams are optimized over the

whole value chain of, and within each, petroleum sector for:

maximum product value

better decisions for investments and technical solutions

cost reductions

Petroleum resources originate as static bodies of oil and gas inunderground reservoirs. As these resources are exploited, they flow aspetroleum streams into a production system. Streams are mixed andprocessed, splitting into new streams. Mixing and splitting of streams

occurs throughout a labyrinth of wells, surface equipment, pipelines,storage facilities, transportation with new mixing and products beinggenerated at the final refinery destination. With Pipe-It users canmanage their petroleum streams and optimize the production ofhydrocarbons.

Pipe-It has the ability to keep a detailed, quantitative upstream-to-downstream accounting of the individual components making uppetroleum resources. This bridges the work of reservoir engineers, whotalks in terms of volumetric rates, to the processing engineers, who

talk in terms of component molar rates, and ultimately to managerswho talk in terms of currency, revenue, net present value and profit.

With Pipe-It software, a petroleum resource stream can be describedsimply as crude oil and gas, or it can be quantified with the hundreds ofmolecular constituents (methane, propane, , benzene, carbon dioxide,). It also allows for every conceivable description between simple andcomplex LPG,NGL,GTL products, crude oil refinery distillation cuts,wax, asphaltenes, and mercaptons. Even monetary value in any

currency, instead of barrels.

Pipe-It allows a project to be visualized with an intuitive graphicallayout design. One visualization can be created to give a clear vision ofthe project from a top-level management point-of-view, showing howfields are connected and interact at major process facilities and exportterminals. Other visualizations detail sub-projects like a subsea welltemplate or a satellite process facility, always maintaining the actualconnectivity to other elements in the project. Custom visualizations ofthe same Project can be made by each individual or team working on a

Project.
http://en.wikipedia.org/wiki/Liquefied_petroleum_gashttp://en.wikipedia.org/wiki/Natural_gas_processinghttp://en.wikipedia.org/wiki/Gas_to_liquidshttp://en.wikipedia.org/wiki/Gas_to_liquidshttp://en.wikipedia.org/wiki/Natural_gas_processinghttp://en.wikipedia.org/wiki/Liquefied_petroleum_gas

8/11/2019 Pipe-It Manaul 20130806

8/264


Pipe-It allows an entire project or elements of a project to be executed(run). This might involve executing external computer programsdescribing performance of reservoirs, wells, pipelines, process facilities,storage and transportation vessels, and final-destination refining. Pipe-It allows these 3rd-party programs to communicate using Pipe-Its

engine.

Not only does Pipe-It rely on computational programs for resourcestreams, it allows access to measured data provided by online meteringand spot testing stored in databases. Sophisticated data managementfeatures of the Streamz engine allows measured and calculated data tobe compared on a consistent basis in terms of quantity (mass, volume,etc.) and time period (daily, monthly, annual). Once on a commonbasis comparative stream data can be used as the basis foroptimization.

A unique technology called Linkz allows Pipe-It to "connect" to tokens(numbers and strings) in text files irrespective of formats. These tokensare then available to Pipe-Its execution and optimization engines andbecome part of the overall optimization. This ability together with thepossibility of executing any program that can be launched from the OSgives Pipe-It an unprecedented capability and flexibility making it anideal framework for integrated computational integration. No programsare imposed on the end user (they choose which programs are bestsuited for their area of expertise), but rather Pipe-It embraces all and

facilitates their integration (connected execution) and data-exchange.

Pipe-Its Optimization Interface allows (through Linkz) any number inany file to be variables, constraints and/or objectives of anoptimization. Multiple levels of optimization can be defined and beactive in a single top level global optimization. Different solvers anddirections of optimization (MAX, MIN) can be specified for each instanceof the global or local optimization. A set of robust solvers are provided,

which can be extended by means of documented API.

Pipe-It includes a command-line version that can be called by not onlyother programs (even batch files) but also by itself! This allows Pipe-Itto be embedded within any other software that can launch an OScommand-line. This also means Pipe-It can recursively call manyinstances of itself from within a top-level graphical instance. This is thebasis of multi-level local and global optimization made possible by Pipe-

It.

8/11/2019 Pipe-It Manaul 20130806

9/264


Getting StartedPetrostreamz Pipe-It(referred to as simply Pipe-Ithenceforth) is asoftware solution for scalable integration and optimization of petroleum

assets. With a unique graphical platform, Pipe-It intuitively integratescomplex data and model configurations typically found in our industry.Once data and models are integrated, Pipe-It provides sensitivityassessment and optimization capability to define business opportunitiesdependent on complex cause-and-affect dependencies of revenueopportunities and cost management. Our fundamental philosophy isthat current work flows should not be disturbed, with each applicationcustodian continuing to use the same software, including all utilities for

pre- and post-processing.

This document provides an introduction to the use of Pipe-It and ismeant to be a guide for understanding and using a project built byPetrostreamz AS for clients. Other sections in this document contain

example to give the users all necessary details.

Please refer to Pipe-ItExample Projectsfor some examples.

Petrostreamz Support department can also arrange necessary trainingcourses, short video tutorials or provide on-site support / consultancy

services. Please [email protected].

Register for an account and a license

To start using Pipe-It, you will need to install the Pipe-It software inyour machine. The software and a trial Pipe-It license is available upon

request.

The first step is to register to create an account. This registration isfree and only requires you to provide some information to be

considered for a three-month trial Pipe-It license. Once you get youraccount, you can sign in to download the Pipe-It software.

The registration procedure:

Go to the websitehttp://petrostreamz.com

Choose "Download" tab on the top menu.

Enter your e-mail address, for first time registration you will need to

choose "No, I'd like to register".
mailto:[email protected]:[email protected]:[email protected]://petrostreamz.com/http://petrostreamz.com/http://petrostreamz.com/http://petrostreamz.com/mailto:[email protected]

8/11/2019 Pipe-It Manaul 20130806

10/264


Click "Go to registration".

Fill-in all required information.

If you are registering to get a trial license, choose "Accept Trial License

terms".

If you or your company has already bought a commercial license,choose "I am a Pipe-It customer and/or my company is a Pipe-It

customer already".

Click "Create new account".

Petrostreamz will send an e-mail confirming that your registration is

successful.

The approval process could take a few days. Once your account hasbeen approved, you will get an e-mail containing information abouthow to log-in, set your password, download Pipe-It and other details.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Register/registration.png

8/11/2019 Pipe-It Manaul 20130806

11/264


Download Pipe-It Software

Once your account at Petrostreamz has been activated, you will receivean email containing all the information you need to log in, set your

password, and other details.

After you log in and set your password by following the procedure

mentioned in the email, you can start to download the Pipe-It software.

How to download Pipe-It:

Log in at http://petrostreamz.com/user using your e-mail and

password.

Navigate to the download page by clicking Download at the top of thepage.

Click on the link provided to access the download area.
http://petrostreamz.com/userhttp://petrostreamz.com/userhttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/DownloadSoftware/download-link.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/DownloadSoftware/download-page.pnghttp://petrostreamz.com/user

8/11/2019 Pipe-It Manaul 20130806

12/264


Once you have navigated to the download area, you can download thePipe-It installation file by choosing the newest version of Pipe-It Setup(.exe). The installation packages for Mac OS X systems and Linuxsystems are available on request.

Note:On the same page you can also download Pipe-It examples andbrochures, these could help you to get familiar with Pipe-It.

Enjoy!

Pipe-It Installation

Windows Operating System

Pipe-It is distributed as a packed installation executable that is anapplication installation and configuration service. This binary is anengine for the installation, maintenance, and removal of Petrostreamzsoftware on modern Microsoft Windows systems.

On other systems the installation procedures might be different.Installation packages are available for Mac OS X systems and Linux

systems (on request).

Quick Installation Guide

Here is an explanation of the installation procedure that allows you to

quickly install Pipe-It on your desktop system.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/DownloadSoftware/download-file.png

8/11/2019 Pipe-It Manaul 20130806

13/264


File Naming and Versioning

Petrostreamz AS provides installation binaries with the following

naming convention, for example:

Pi peI t Set up- 1. 3. 0. exe

This allows you to keep track of build number and have up-to-dateversions. You can easily compare this build ID with that of the versioncurrently installed, which is shown in the About dialog (About Us on the

Help menu):

Executing Pipe-It Installer

Welcome Message

The first screen that you see indicates that Pipe-It installation issuccessfully started and that it will guide you through all installationsteps and choices. It also shows the build number to keep you aware of

which version you are installing:

8/11/2019 Pipe-It Manaul 20130806

14/264


Choose Install Location

This page allows you to check the installation path of Pipe-It and if

required change it to another one that is more convenient for yourinstallation. Note that this installation path is stored in the registry, sowhen you run a Pipe-It installer with a new version to update Pipe-It, itwill load the previous installation path automatically. Then you only

need to check that it is correct and press the "Install" button:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItInstallation/install-wizard.png

8/11/2019 Pipe-It Manaul 20130806

15/264


Installation Progress

The page indicates installation progress, and shows you the files beingcopied to your hard drive if "Show details" is clicked. Just wait fewseconds to get the software installed:

Installation Completed

Finally you get a screen that indicates that the installation iscompleted. Also by default it allows you to run Pipe-It directly when the

installation is closed:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItInstallation/install-completed.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItInstallation/install-progress.png

8/11/2019 Pipe-It Manaul 20130806

16/264


Installed Components

The Pipe-It software product includes the following components

distributed within the installer binary:

Petrostreamz components

Pipe-It.exe- Main GUI Application.

Pipe-Itc.exe - Non-GUI version of Pipe-It that can be invoked from

other programs or by Pipe-It itself!

Streamz.exe - The "Engine" of Pipe-It. Sophisticated streammanagement program with track record of industry-strength projects

under its belt. See Streamz documentation for more info.

ProjectBuilder.exe - Wizard-based builder of Pipe-It project. Uses atemplate (see below) designed for specific application (e.g. EclBOz) andcustomizes it based on user supplied data. The Wizard displays pagesfor user interaction and replaces/duplicates elements in the template to

create a ready to run Pipe-It project.

Growing set of standard conversion utilities to facilitate integration of

popular engineering software (e.g. Eclipse, Sensor)

A set of Solver for use by the Optimizer

A set of Project Builder Templates

A set of Scripter Assistant Program short-cuts.

3rd party components

Qt Libraries- DLLs to run GUI software.

UnxUtils - Set of Unix utilities built for Win32 to use in projects forcross-platform issues. See also http://unxutils.sourceforge.net/ Theinstaller program appends the path in each launched instance of Pipe-itso all these commands are available in command line.

Pipe-It Preferences

Petrostreamz Pipe-It, like most of the applications on your computer, hasa set of preferences you can customize. You use preferences to specify
http://unxutils.sourceforge.net/http://unxutils.sourceforge.net/http://unxutils.sourceforge.net/

8/11/2019 Pipe-It Manaul 20130806

17/264


things such as auto-save settings and a default folder for storing projects.At this point it is recommended to change at least the default editor fromNotepad (on Windows) to the one you are used to. You can worry aboutthe others as you get more experience with using Pipe-It.

To view or change preferences:

Win: Choose Edit > Preferences

Mac: Choose Pipe-It > Preferences

Preferences are tabbed into 3 pages:

General

Text Editor

Canvas

General Preferences

The page of preferences defines settings:

Beep on project run finished- the checkbox makes Pipe-It beeping whenexecution of your project is finished. By default the setting is enabled, butyou can disable it when running Optimizations and it produces severalbeeps too often. Anyway this is good indication for long term executions

to attract attention.

8/11/2019 Pipe-It Manaul 20130806

18/264


Auto-save opened projects - the option with defined interval enablestimer to save your project after defined time from last save operation. Itsaves project into model+view files, ppm and ppv with same file name asopened project but prepended with "~". In case of power failure you havepossibility to extract lost changes to project by getting information from

auto-save files.

Text Editor Preferences

Text Editor page allows you to define default text editor for Edit Itemaction for Resources. Note that it is different than double-click onResource which invokes Application from default system association withthis file type. Instead Right click on Resource and choosing Edit Item, or

just pressing F4invokes association defined by this preferences page.

It can be very useful if you would like to have fast way of starting Fileedition process but without changing the default association in OS to starteditor with double-click. Using F4 invokes Editor specified in Pipe-It andallows to keep double-clicking for processing the file by some utility.

You can see that the page allows you to extend associations for severalfile types. By default it provides additional association for image files to

show you example of defining new bindings to your editing applications.

Canvas Preferences

You can customize next canvas settings:

8/11/2019 Pipe-It Manaul 20130806

19/264


Use mouse wheel for zooming By default the mouse while (or thetrackpad found on current portable computers) is used to scroll up anddown in the canvas. This setting will allow the user to use it for zoomingin and out instead.

Reverse mouse wheel direction when use for zoom Zoom-in and Zoom-

out are swapped from the default action.

Animate connectors on Project Run - when you run your project, thenactive connections (which are connected to elements that are beingprocessed this time) are animated by moving dots to show you which

Resources are currently used or produced by running Scripters.

Make new item selected when adding to canvas Fairly obvious.

Make green checkmarks disappear on first mouse move- When executionof project is completed then all processed elements got checkmarks (inright top corner) to indicate completeness of executed elements.

Checkmarks that indicates problems attracts attention to resolve situationand start execution again. But in case of successful execution all elements
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ApplicationPreferences/AppHelpPreferencesDialog-AnimateConnectors-1.png

8/11/2019 Pipe-It Manaul 20130806

20/264


are marked with green checkmarks and not required to be kept on canvasto continue your operations. For such case you can use the option to keepgreen checkmarks until first mouse move - you definitely notice theproject state and can continue work.

It is convenient if project run time is huge and you need indication whenyou return to computer to look at completeness of the execution.

Animate slow shading during Time used to fade the checkmarks.

8/11/2019 Pipe-It Manaul 20130806

21/264


How to use Pipe-It: A small workingexampleYou can use Pipe-It to "pipe" together your project from bottom-up ortop-down. Each software application being launched is called a Process.Any file connected with a Process is called a Resource. You can chaintogether as many Processes as needed using Connectors, in any order(parallel and/or series). A collection of Processes can be grouped into aComposite. The total collection of all Processes, Resources, andComposites is a Pipe-It Project. An entire Pipe-It Project can be runtime-and-again with changes to any input data using the Optimizer.The Runner orchestrates, without user intervention, intelligent andefficient launching of all Processes in a Pipe-It Project. And by usingLinkz,any data in Resource files can be accessed and used for further

operation.

In this page, you will find a step-by-step procedure to build a verysimple Pipe-It project. This example will help you to start theinteraction with Pipe-It. A detailed description of a Pipe-It element can

be examined by clicking the referenced item (a hyperlink).

Pipe-It Example: Launch a GAWK program

In this example, we will utilize Pipe-It to launch a gawk program (http://www.gnu.org/software/gawk/manual/gawk.html) that willextract data from a text file. Windows version of GAWK is installed withPipe-It as part of UnixUtils, to facilitate execution of Pipe-It projects

across operating systems.

Open the Pipe-It software. Go to the Start menu and choose the Pipe-It

program.

Save the Pipe-It project in your preferred directory.Go to the File menu and choose "Save" or "Save As". You could also

use the shortcut provided (Ctrl+S or Ctrl+Shift+S) to do this operation.

Note:A shortcut for an operation (if any) can be seen on the menu.
http://.http/www.gnu.org/software/gawk/manual/gawk.htmlhttp://.http/www.gnu.org/software/gawk/manual/gawk.htmlhttp://.http/www.gnu.org/software/gawk/manual/gawk.htmlhttp://.http/www.gnu.org/software/gawk/manual/gawk.htmlhttp://.http/www.gnu.org/software/gawk/manual/gawk.html

8/11/2019 Pipe-It Manaul 20130806

22/264


You can start building the project by adding a title or a note in theCanvas to identify and/or give a general description of the Pipe-It

project.

Choose the text-box (Annotation) symbol in the Toolbar and place it at

the preferred location on the Canvas. Double-click to edit theannotation text and press "OK" when finished.You could also edit the display (Color/Line/Font etc.) by placing thecursor on the text-box, right-click, and choose "Open Item

Properties...".
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-textbox.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-save.png

8/11/2019 Pipe-It Manaul 20130806

23/264


Adding a Resource to the Pipe-It project. Choose the Resource symbolon the Toolbar, or just simply press "R", and place a Resource element

on the Canvas.

Associate a data-file (employee.txt) with the Resource.Right click on the Resource box and choose "Set Resource File...". Findthe data-file and click "Save". Un this case you will create this file, so

just give it the suggested, or another, name.

Right-click this resource and choose to Open in Editor (or just press

F4 after selecting the resource:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-resource-file.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-resource-toolbar.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-textbox-itemprop.png

8/11/2019 Pipe-It Manaul 20130806

24/264


Depending on your Pipe-It Preferences the empty file will open inNotepad or your preferred text Editor. Populate it with the following

data and Save it.

00 Thomas Manager Sal es $5, 000200 J ason Devel oper Technol ogy $5, 500300 Sanj ay Sysadmi n Technol ogy $7, 000400 Ni sha Manager Market i ng $9, 500500 Randy DBA Technol ogy $6, 000

This data can be copied from the website which is the source of thisexample:

(http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-

awk-print-examples/)

Add one more Resources and associate it with the (actions.awk) using

the same procedure as above and populate it with the following:

{pri nt ; }

Adding a Process to the Pipe-It project. Choose Process symbol on theToolbar and choose "Program(Script)" or just simply press "P". Placethe Process element on the Canvas.
http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/

8/11/2019 Pipe-It Manaul 20130806

25/264


You can rename the Process element by right-clicking on it andselecting "Rename Element...", or just simply press F2 with the Processelement on the Canvas selected.

Connect all input resources to the process by using Connectors.Click the Connectors symbol on the Toolbar to activate it and thenconnect the preferred resource with the process element. First movethe mouse to the (actions.awk) Resource, connection points (smallwhite marks) should show up on the edge of the element. Left-clickonce at one of the points, then move to the Process element and left-

click again at one of the connection points on that element (you canalso just click and hold the left mouse button at the connection point onthe Resource and then go to a connection point on the Process andrelease the button). Now the Resource and the Process should beconnected by an arrow, going from the Resource to the Process. Do the

same for the (employee.txt) Resource.

The Pipe-It project should now look similar to this. A second annotation

(text box) has been added to provide additional information.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-process.png

8/11/2019 Pipe-It Manaul 20130806

26/264


To be able to launch the Gawk program, we need to write a scriptinside the Process element based on the Pipe-It usage of the AWK

syntax.

Double-click the process element and in the "Script" box type thecommand line to invoke Gawk application.

gawk

The syntax of the awk command is actually:

gawk - f act i ons- f i l e dat a- f i l e [ > resul t - f i l e]

Thus, inside the "Script" box we need to use the syntax above and insert

actions-file (.awk) first, then the input file and the output file.

The command line inside the "Script" window should look like:

8/11/2019 Pipe-It Manaul 20130806

27/264


Run the Pipe-It project by clicking the "Run Project" symbol on the

Toolbar or by using the shortcut "Ctrl+R".

A green check-mark symbol will appear on the elements if the projectruns successfully. If an error occurs, you will see a red-flag symbol on thebroken element.

Since the optional output file was not specified in this example, gawksends the output of the program to the standard output. Pipe-Itprovides a console to capture this output from the program. To view

this console click on the View menu item and select Show Console:

A Console Window will open to show the result of the execution. In thisexample since the actions file contained only the action to print, each

line will be printed on the console:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/PipeItExample1/example-run-project.png

8/11/2019 Pipe-It Manaul 20130806

28/264


The next step in this mini-tutorial is to save the result instead to a file.Create another Resource on the canvas and connect it as an Output tothe Process (start the connector from the Process and drive it to the

new Resource). Associate this resource to a file named result.txt:

Canvas

Console

Output of gawk

8/11/2019 Pipe-It Manaul 20130806

29/264


Since the syntax of gawkrequires redirection operator (>) to passthe output to a file instead of to the console, we need to modify theScript in the Program. To enable the > to be interpreted correctly bythe operating system (Windows), the command line needs to beprocessed via the command shell. The final command needed looks like

this:

Run the project again. This time the Console only logs what was donebut the result is sent to the file. In the picture below the result.txt file

is opened in Notepad to show the content.

8/11/2019 Pipe-It Manaul 20130806

30/264


Pipe-It offers many features to make your projects flexible, robust andsustainable. One such feature is the use of Resource Referencesinstead of directly coded file names (Covered in detail later in thismanual). You might want to come back to this project when you havestudied it. The screenshot below shows the content of the Process. Note

that the Script uses Resource References syntax while the parsedcommand line is identical to what we saw earlier.

Before we close this project, let us try to alter the action, which is veryboring right now. Change the content of the action.awk file to (fourthcol umn cont ai ns Technol ogy) :

$4 ~/Technology/ {print;}

Resource reference syntax

Corresponding

Command line

8/11/2019 Pipe-It Manaul 20130806

31/264


Pipe-It uses gawk to select only lines fulfilling certain criteria and write

the results to a file. Save and close the Pipe-It project.

Congratulations! You have just created your first Pipe-It project and

run it.

This Pipe-It project example is very simple and far from what Pipe-Itcan do in real application. This exercise comes with the intention to

help new users to start the basic interaction with Pipe-It.

For some real Pipe-It project examples, please refer to Pipe-It Example

Projectsand on the website.

8/11/2019 Pipe-It Manaul 20130806

32/264


Help and DocumentationThere are several ways to get help in Pipe-It, depending on the level of

help required.

Tooltip

Hypertext - short description of object in the user interface. Thedescription usually appears within a few seconds when the cursor is

placed over an object.

Pipe-It Help System

Pipe-It comes with a built-in help system where you can find Pipe-Itand Streamz documentation (in English). You can invoke this systemby pressing F1 or going to the Help menu in the main Pipe-It window.

Currently this PDF Pipe-It Manual is linked to the help system.

Future versions will be provided in a format similar to a web browserthat shows hypertext content with links and images. By using thissystem, you can access the Pipe-It and Streamz documentation, apanel with a list of keywords, a bookmarks panel and a search system

that allows you to easily find the content that you are interested in.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/HelpAndDocumentation/help-tab.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/HelpAndDocumentation/tooltips.png

8/11/2019 Pipe-It Manaul 20130806

33/264


Support by email

Sending an e-mail - to the Support Department at

[email protected] the highest level of help and can be usedto resolve intricate problems when you are otherwise stuck.
mailto:[email protected]:[email protected]://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/HelpAndDocumentation/help-system.pngmailto:[email protected]:[email protected]:[email protected]

8/11/2019 Pipe-It Manaul 20130806

34/264


The main software components

Introduction

This section will give a brief introduction to the software components in

the Pipe-It package.

The figure shows the main software components of the Pipe-It package.In the figure, solid lines indicate that the program has a graphical userinterface (GUI). Dashed line indicates that the program is started fromthe command line, without GUI. File type extensions of the input filesare also shown. Most of the programs write out to the same file or

same type of file.

Pipe-It

The main program with GUI.

Pipe-Itc

The command line version of Pipe-It.

Linkz

It is used to define links to individual data elements or matrices insidefiles. Linkz can work with any text file and with MS Excel (binary orxml) files. Through a GUI the user can "teach" Linkz how to find the

desired data elements. The link definitions are saved in a .ppl file.

Optimizer
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/MainSoftwareComponents/MainSoftwareComponents.png

8/11/2019 Pipe-It Manaul 20130806

35/264


A control panel to set up and start automatic iterations of executions,using a solver. The optimizer can use links into data files defined with

Linkz.

(Solvers)

A solver contains the optimization algorithm. It is started by theOptimizer and takes over control during optimization. A collection ofsolvers is delivered with Pipe-It, most of them are in dynamic libraries

(.dll). Choice of solver is available from a menu in the Optimizer.

Runner

A part of Pipe-It that takes care of execution of the project. It is notvisible to the user. The Runner runs only the parts of a large project

that need updating according to changed data and dependencies.

Streamz

Uses .str (Stream files). It calculates combination and splitting ofcollections of streams. It is both a command line program and tightlyintegrated within Pipe-It.

Strexzel

A spreadsheet-like program that is specially made to do operations on.str files, which are beyond the scope of Streamz.

Plotz

Makes graphs / plots.

MapLinkz

Copies data from one file to another by using links defined with Linkz.It is integrated with Pipe-It and easy to use in projects in the Pipe-It

GUI.

(Converters)

A collection of utilities that can read various other file formats,including binary files, and produce .str files.

ProjectBuilder

A utility program that can read definitions from "templates" (made by

expert users), and based on these it shows Wizard-like question pagesto the user, collects data about the new project from the user's

8/11/2019 Pipe-It Manaul 20130806

36/264


answers, and generates (possibly very large) Pipe-It project files as a

result.

ScriptAssistant

A utility that helps the user to set up how to use (a new) externalprogram (what input it needs, what output it generates etc.) fromwithin Pipe-It. The result is that the external program becomes

apparently integrated in Pipe-It and easier to use in a Pipe-It project.

Terminology

Unsure about what we mean? Need a one line definition of a term used onthe website? What is a Resource? A Process?

Refer to the Appendix for a collection of all technical terms in the contextof Pipe-It. Even Pipe-It is defined here! Use the search function to search

for the term.

8/11/2019 Pipe-It Manaul 20130806

37/264


Pipe-It Graphical ElementsPipe-It is a unique software application that allows you to graphicallyand computationally integrate models and optimize petroleum assets.

Pipe-It can launch any software application you are currently running,on any operating system. Pipe-It chains together your applications, inseries and parallel. Pipe-It's versatile Streamz technology manages thecomplex oil and gas streams processed by models making up anintegrated asset. Our fundamental philosophy is that current work flowsshould not be disturbed, with each application custodian continuing touse the same software, including all utilities for pre- and post-processing. The main function of Pipe-It is to provide added value

through

modular & scalable integration solutions

local & global optimization capability

visual organizational functionality

Resources

Resources are representations of files on disk. The default shape is arounded rectangle with a pale blue color. They should usually berenamed to something appropriate in the context of the project. Theyneed to be linked to the appropriate file on disk via "Set Resource file"context-menu option. An easy way to create a new (or multiple)resource is to drag the file from file explorer on to canvas. This createsthe resource, gives it a default name and also set the file linking in on

quick step.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Resource/resource.jpg

8/11/2019 Pipe-It Manaul 20130806

38/264

8/11/2019 Pipe-It Manaul 20130806

39/264


The element can be renamed to a more convenient name by pressingF2 and typing in a new name. It will still be associated with theselected file.

If the resource points to a stream file,a characterization name can be

associated with the stream file by pressing F3 and typing in the name.

Pressing + will open the item properties dialog, where thevarious properties can be inspected and modified. If characterizationshave been added to the project via Streamz Library(), these will be available in the Characterizationdropdown. Stream file input and output options can also be defined in

this property page.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Resource/itemproperties1.jpghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Resource/setResourceChar.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Resource/resourcenamechanged.jpg

8/11/2019 Pipe-It Manaul 20130806

40/264


Program (Script)

Programs are operations that modifies an upstream resource to createa new downstream resource . The default shape is an ellipse (oval) with

a pale green color. They should usually be renamed to somethingappropriate in the context of the project. They are usually connected onthe upstream and the downstream sides to resources. They can neverbe connected directly to another program. The most often usedprogram is a generic Scripter where the user can specify any commandline for execution. A program canvas element can be created byclicking on the "P" icon in the toolbar and dragging the ellipse on to the

canvas.

The application executable and its command line arguments isdescribed in the scripter properties dialog. Select the process and press

to open this dialog, or open it from the context menu.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/processcontextmenu.jpghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/process.jpg

8/11/2019 Pipe-It Manaul 20130806

41/264


Scripter Dialog Box

Example

The scripter command line can be explicitly entered by the user. Forexample if the scripter launches the reservoir simulator SENSOR andfirst argument is the input data file, the commands written into the

scripter would be:

SENSOR. exe i nput . dat

Pipe-It allows a flexible syntax to make projects and programs portableby using reference to connected resources. The files actually connectedto those resource will then be used to make up the command line on-

the-fly at time of execution.

To illustrate this, let's set up a process for converting a stream file withthe Strexzel application, we will need an input stream file, a macro file.

When the program executes, it will produce an output file.

Let's select the program and open the scripter properties dialog. Youcan do this by selecting the program and by pressing F4, or by

selecting "Open Scripter" from the context program context menu.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/OpenScripter.jpghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/process_example.jpg

8/11/2019 Pipe-It Manaul 20130806

42/264


You get a window divided into 3 parts. The first part with the labelScript is a space for you to write the command required to launch

your program.

In the example below, we have used the Strexzel utility. A normalcommand line invocation for Strexzel for opening a stream file, runninga macro file and output the result to a second stream file is:

St r exzel - m - i - o

First we need to type in the name of the application that we want Pipe-It to execute when the project runs. In this case, the application nameis "Strexzel". The simplest way to populate this script area is directlyfile names. However the user needs to know exactly the names of thefiles (including the path relative to the current folder). Suppose the filesare located in the current folder. The following picture shows what a

Script may look like.

8/11/2019 Pipe-It Manaul 20130806

43/264


In this case the Script and the Command Line are identical.

However Pipe-It is designed to provide the names of the files needed asarguments on the command line automatically, if a Resource (set to afile) is connected to the Process using a Connector. In fact that is thepurpose of the connectors to provide Processes with files (via

Resources).

To use this flexible, and more robust, way of populating the Script youneed to use the Insert New Argument button. This results in a box

with many choices. Many of the defaults can usually be used.

8/11/2019 Pipe-It Manaul 20130806

44/264

8/11/2019 Pipe-It Manaul 20130806

45/264


Where clicking the button of your choice will select it and take you back

to the original window.

If the program had been created using a utility, or a user has added

more sockets, a list of recognized sockets would have been displayed.

Since we are working with a simple example with two input files andone output files, we proceed with the default sockets and stick with I1

(First Input Socket).

We need to set the resource that will be used as the first argument. Weclick on the Choose button in front of First Resource to get the

following dialog:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/resourceReferenceDialog.jpghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/socketReferenceDialogOpen.jpg

8/11/2019 Pipe-It Manaul 20130806

46/264


This dialog lists the resources that are connected to the chosen Socket(I1 in our case) of the program. We can select the resource usingforward referencing or backward referencing indexes. In the forwardreferencing scheme, an integer index is assigned to each resource. Inthe backward referencing scheme an integer reference relative to the

number of resources is used. If we have three resources and want toreference the second one using the forward referencing scheme, we willuse the index "2". If we want to use backward referencing, we will use

the index "n-1".

Backward referencing is for geeks. Normal people will use forwardreferences. In this example we will use forward referencing. Since wealready have the resources (with files) connected to the Program, theyshow up under respective columns. We select the macro file by clickingon its index (1) in the forward referencing column. The dialog will close

automatically after we have selected a reference.

Since this argument also needs a switch (-m) we can specify it as part

of the Bounding Text entry as shown in the figure:

This ensures that the switch is written only if the argument is written tothe command line (useful for optional arguments). When you click

Insert you get a new entry in the script.

8/11/2019 Pipe-It Manaul 20130806

47/264

8/11/2019 Pipe-It Manaul 20130806

48/264


Summary of Scripter Dialog Options

Field Mandatory Description Actions / dialogs

Script yes Describes an applicationcommand line using

Pipe-It referencesyntax.

edit manually orcompose by using the

"Insert newreference" button.

Socket yes socket name to beinserted in reference.This can be an input oroutput socket

Edit manually or openthe "socket reference"dialog by pressing the"choose" button nextto the field. Thisopens the socketreference dialog(described below).

Selecting a value inthis dialog will

8/11/2019 Pipe-It Manaul 20130806

49/264


populate the field.

Firstresource

yes The first resource nameto be inserted in thereference

Edit manually or openthe "resourcereference" dialog by

pressing the "choose"button next to thefield. This opens theresource referencedialog (describedbelow). Selecting avalue in this dialogwill populate the field.

Lastresource

no This lets you select aresource as above, but

it is only used if youwant to specify a rangeof resources in thereference. The range isgiven by two indexesusing forward orbackward referencing.

same as above.

Minimum no The field "minimum"(matches required forinsertion) indicates the

minimum number ofconnected resourcesthat has to be active forthe program to execute.

Spin button forincreasing ordecreasing the value.

Relative /Fixed filepath

yes Defines the reference asa fixed path or arelative reference

Select using radiobuttons. Defaults torelative.

ExcludeFile

Extension

no If this is selected, thereference expansion will

exlude the file extension

Checkbox.

ParentDirectoryOnly

no If this is selected, thereference expansion willonly refer to the parentdirectory for theresource

Checkbox.

Insert yes Press this button toinsert a new referencein the script field. The

reference will becomposed using

Button.

8/11/2019 Pipe-It Manaul 20130806

50/264


selected values from thefields described above.

Error - Area to show whenScript has un references

resources/filesWorkingdir

no Defines the workingdirectory for theprogram.

Pressing the buttondisplays a field wherethe working directorycan be defined.

Utility no Expands a list withpreconfigured utilities.Selecting a utility fromthe list will preconfigurethe contents.

The button opens thescript translationdialog (describedbelow).

Notice that Pipe-It has preconfigured Program templates (utilities) for aset of bundled applications. By using these to define a program insteadof the generic Program icon, Pipe-It will be able to provide a full set of

possible command line parameters.

Each argument in the Script is abstracted as follows

The first character in the argument specifies if the resource is

referenced as a relative or a fixed path.

"R1@I1" signifies that the resource path is relative "F1@I1" signifies thatthe resource path is fixed

The second character refers to the resource index. This is used toidentify the correct resource in the case that multiple resources are

connected to a program via the same socket

"R1@I1" signifies that the resource path is relative and that we reference

the first resource
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/StrexzelUtility.jpg

8/11/2019 Pipe-It Manaul 20130806

51/264


Notice that Pipe-It uses its own abstractions for representing commandline arguments. The command line switches and options are identical toa normal command line, but the argument file references are replacedwith an alternate syntax. This syntax makes it possible for Pipe-It tocreate valid program command lines by referring to its own resource

abstractions instead of referring to specific files directly. By creating aprogram representation with the Strexzel utility, Pipe-It creates thefollowing command line for us:

St r exzel { - m "R1@I 1" } { - i "R1@I 2" } { - c "R1@I 3" } { - o "R1@O1" } {- s "R1@O2" } { - l "R1@O3" }

This predefined command line contains all known command linearguments for the applications.

To illustrate this, let's set up a process for converting a stream file with

the Strexzel application, we will need an input stream file, a macro file.When the program executes, it will produce an output file.

Since Pipe-It is aware of the command line interface for Strexzel, The

command line

St r exzel { - m "R1@I 1" }{ - i "R1@I 2" }{ - c " R1@I 3" } { - o "R1@O1"}{- s "R1@O2" } { - l "R1@O3" }

is generated automatically.

Connectors pointing to the program are interpreted as input, andconnectors pointing away from the program are interpreted as output.Resources are referenced via input or output sockets. Sockets are

identified by type and number
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Pipe-It_GUI_Program/process_example.jpg

8/11/2019 Pipe-It Manaul 20130806

52/264


Pipe-It Projects & Sub-ProjectsExcel files are called spreadsheets. Powerpoint files are calledpresentations. Similarly Pipe-It "files" are called projects. Pipe-It

projects consists of many component files. Not just single files likemost other applications. Pipe-It Project View (*.ppv) file store thevisual content of the project. Pipe-It Project Model (*.ppm) file is mostimportant and stores the logical model describing the all the elementsin the project their connections, their grouping into composites, thefiles associated with each resource, the command lines of each scripteretc. Pipe-It Project Linkz (*.ppl) file stores all the variables collectedfrom all text files that are part of the project, their current values andalgorithms to locate them in later runs. Pipe-It Project Streamz Library(*.pps) files store Streamz characterizations and conversions used in

the project.

The location of these 4 files is called the root folder of the project. Mostother files mentioned under can be located anywhere on the disk.However, to make the project portable it is recommended that all such

files are located at or under the root folder.

If the project contains optimization, one or a series of Pipe-It ProjectOptimization (*.ppo) files are also present. They can be located

anywhere.

Since Pipe-It is a multiple application integration framework, allowingthe launch of many internal and external applications, a large numberof application specific files become part of the project. Many of thesefiles may be represented as Resources in the Pipe-It project andbecome "known" to the project. Many other files just exist and arecreated, read and written by applications. All files connected toinitiating resources (those that are not downstream of any process) arealso mandatory for the proper execution of the project. These files aretypically located at the root or in folders under the root.

Designing a Pipe-It project requires some planning. Simple projectsmight contain only a handful of data files linked to resources and thesecan live in the project root folder. But real life projects containhundreds if not thousands of files. It makes sense to organize thesefiles in a sensible folder hierarchy. The project within Pipe-It shouldalso follow the folder structure using composites and sub-composites asfar as possible. This helps in cloning of composites in case a newinstance of the functionality encapsulated in the composite is to becreated. Pipe-It estimates the % of files that resides within a particularsub-folder of the project and offers to the user the possibility of

duplicating files. This is really a powerful option as all files, their

8/11/2019 Pipe-It Manaul 20130806

53/264


connections to resources, their Linkz variables, and all process

command lines are cloned.

Making order of the hundreds and thousand files is another reason for

collecting related files together.

Projects & Composites

Project is container for all elements of modeling processes. Of coursekeeping all the elements on same canvas makes the scheme verycomplex for understanding and tracking processe details. To help withthis some or all elements in a project can be organized in Sub-

Processes or Composites.

We recommend to use the term Projectfor meaning whole project. Alsoit means a set of 4 files that defines the project - logical model + visual

model + variable links + Streamz Library (ppm + ppv + ppl + pps).

The term Composite is used for defining some part of whole projectthat is organized separately from other elements. Actually compositesare small sub-projects that work in same way as other elements of

canvas.

Because Composite is actually sub-project that defines someoperations, it can also have nested composites of second level and so

on.

From this point of view, the project is tree of composites:

Or similar but top to bottom structure:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Projects/TreeOfComposites-1.png

8/11/2019 Pipe-It Manaul 20130806

54/264


Composites

As mentioned in the section on Projects, composites are a way to makea project easier to understand. They will compose parts of a projecttogether into one box for a less cluttered appearance. For example onemay make one composite for the reservoir part of the field, and one forthe processing part of the field. Each composite may also be run

individually. Project can have any amount of nested composites:

Connections from elements within a composite to elements outside (orinside other composites) are made viaSocketsthat look visually unique

for each different type.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Projects/TreeOfComposites-2.png

8/11/2019 Pipe-It Manaul 20130806

55/264


Export of Composite

It is possible to export complete composites to standalone projects by

using:

Selecting the Composite and right-clicking give this dialog. There are

two possibilities:

Save Composite as a Pipe-It Project: The data files connected toresources are referenced at original locations. All nested elements are
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Composites/CompositesConnections-1.png

8/11/2019 Pipe-It Manaul 20130806

56/264


carried over. But associated Linkz variables and Streamz Library are

not.

Save Composite as a Pipe-It Archive creates a complete standalonecopy of all datafiles, folders, files, Linkz variables, Streamz Library that

are specific to this composite, and nested composites. This is apreferred way to send a complete portion of a project to someone else.

Importing

Importing composites is done Selecting the File/Import menu optionafter which the typical file browser window allows you to find the ppv

file to import.

An alternate way is to drag the ppv file, like any other resource, on to

the canvas. In this case a dialog to choose insertion of resource orimport of composite is offered:

8/11/2019 Pipe-It Manaul 20130806

57/264


See Also

Sockets

Socket Types

Sockets

Except for connectors, every element has a number of sockets. Whenyou connect two elements, you actually connect an output socket fromone to an input socket on the other. Each socket has a differentpurpose. Think of connecting a stereo receiver to a set of speakers. You

can't just attach your wires at random. Each speaker will have exactlytwo terminals (sockets), but the receiver might have 20 or 30, all witha different purpose. You have to connect the receiver's "Right SpeakerPositive" terminal to the positive terminal on the right-hand speaker,

and so forth.

All currently defined resources, manifolds, and distributors have exactlytwo sockets -- one for input and one for output. By default, they'renamed Input and Output, respectively. You're free to rename them(except Resources), but you can't delete these sockets or add newones. Currently defined processes (i.e., Scripters), also start with twosockets, Input and Output, but you can add any number of either type,as well as rename them. Since you can define your own sockets foreach process, it's up to you to define how they're to be used and towhat they should be connected. Generally, though, if you plan to attachresources of different types to a process, it would make sense to definea different socket for each type of input or output.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/Sockets/SocketsExample1.png

8/11/2019 Pipe-It Manaul 20130806

58/264


From the contextual pop-up menu, bring up the Edit Socketsdialog forthe Scripter. There you'll see all of your sockets and be able to editthem. You'll also see which connectors are attached to each socket andin which order (which is also editable).

TIP: If different "types" of resources are connected to a scripter andare to be used for different types of arguments on the command linethen it is recommended to create new sockets for each type ofconnected resource, otherwise the wrong file may end up at theposition of the wrong argument.

Socket Types

All elements (except connectors) have sockets. Let's pass through

types to show what kinds of sockets can be used in appropriate items.

Resources

Resources have exactly two sockets -- one for input and one foroutput:

Resource Input Socket

Resource Output Socket

ProcessesScripters also start with two sockets, Input and Output, but you candelete either one and/or add any number of either type, as well asrename them.

Process Input Socket

Process Output Socket
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/SocketTypes/SocketTypes-1.png

8/11/2019 Pipe-It Manaul 20130806

59/264


Composites

Composites can have all 4 types of sockets due to all available internal

elements:

Resource Input Socket

Resource Output Socket

Process Input Socket

Process Output Socket

Let's illustrate all possible types of sockets inside composites:
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/SocketTypes/SocketTypes-3.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/SocketTypes/SocketTypes-2.png

8/11/2019 Pipe-It Manaul 20130806

60/264


Runner

The Runner is a behind-the-scene component of Pipe-It thatorchestrates the execution of the project. It checks the status of all the

elements, if all the resources have associated files connected, anddecides the order of execution. It manages the launching of scriptercommand lines to the OS and waits for their completion. It determinesthe complete dependencies of the elements and keeps the elements onhold till all the upstream elements are up to date. The runner is visually

invisible to the end user, who will only interact through the GUI.

Pipe-Itcallows running a model from a command line without invokingthe GUI at all. In such case the Pipe-It Runner is only being invoked

directly.

Runner's Project (Composite) execution logic

The rules for running composites are the following:

All active processes within the running composite, and within itsnested, active sub-composites, will be checked and run in the proper

order.

All active resourceswithin the running composite, and within its nested,active sub-composites, will be checked and updated in the proper

order.

All processes outside of the running composite will be consideredinactive (as will any connection to or from such a process).

All active resources outside of the running composite will be checkedand updated if attached to the running composite by a chain of active

connections that is either strictly upstream or strictly downstream.

If one wants to completely isolate a running composite from all outside

elements, the easiest way to do that is to deactivate the compositeitself (non-recursively). The composite can still be opened and run inthat state, but all outside connections will be considered inactive. Thiscould prove to be a useful technique for testing individual composites.Be advised that the results might differ from those obtained by running

the complete project, however.

Using Projects by multiple users

Pipe-It allows projects to be opened by single user at same time. Toprevent inconsistency due to simultaneous modifications to project

8/11/2019 Pipe-It Manaul 20130806

61/264


files, there is lock file is created when user opens a project. The lockfile contains information about user and host name, time stamp of last

access (load or save) to project and PID of process.

When you try to open project being in use by another user, you will get

a warning which indicates a user name and computer, so you cancontact him directly to unlock the project and have possibility to open

and modify it.

Example Pipe-It Projects

This section gives glimpses of the showcase projects built with Pipe-It.Both the scale (number of elements) and technical complexity and

variety of the projects are impressive.

Multi-Field Complex Fluid Integration (Statoil)

Over 11,000 Resources in this project and nearly 3500 Processes

(application launches).
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/MultiUsersProjects/AppHelpLockFileWarning-1.png

8/11/2019 Pipe-It Manaul 20130806

62/264


Production Optimization with Assisted HM (Petrobras)

Master project that performs the production optimization.

Compositional Well Test Allocation (Equion/Ecopetrol)

Almost 1500 Resources and 750 Processes organized in 600

Composites.

8/11/2019 Pipe-It Manaul 20130806

63/264

8/11/2019 Pipe-It Manaul 20130806

64/264


If the project contains optimization, one or a series of Pipe-It ProjectOptimization (*.ppo) files are also present. They can be located

anywhere.

A complete project will also contain data files that are connected to

each resource. All files connected to initiating resources (those that arenot downstream of any process) are also mandatory for the properexecution of the project. These files are typically located at the root or

in folders under the root.

Designing a Pipe-It project requires some planning. Simple projectsmight contain only a handful of data files linked to resources and thesecan live in the project root folder. But real life projects containhundreds if not thousands of files. It makes sense to organize thesefiles in a sensible folder hierarchy. The project within Pipe-It should

also follow the folder structure using composites and sub-composites asfar as possible. This helps in cloning of composites in case a newinstance of the functionality encapsulated in the composite is to becreated. Pipe-It estimates the % of files that resides within a particularsub-folder of the project and offers to the user the possibility ofduplicating files. This is really a powerful option as all files, theirconnections to resources, their Linkz variables, and all process

command lines are cloned.

Making order of the hundreds and thousand files is another reason for

collecting related files together.

Use Recycle Loops

Pipe-It runs projects in an efficient manner. Only outdated parts areexecuted. All resources in the project are checked for modificationdates of files to decide which part of project should be executed. Thisavoids unnecessary execution of the same Process in subsequent runs

to generate identical data.

But there may be certain instances when some identified portion of the

project should be executed each time you press Run button. Forexample, imagine a project which fetches input data from some
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/UseRecycleLoops/AppHelpUseRecycleLoops-1.png

8/11/2019 Pipe-It Manaul 20130806

65/264


network services (e.g. Web Services). Before processing the rest of theproject there should be a mechanism to get updated data from theexternal component (Web Services) since the external component maynot be automatically updating a resource within the Pipe-It project.

A simple two-manifold recycle loop is the simplest way to achieve this.Everything within and downstream of such a recycle loop is guaranteedto run every time the project is run (because the loop always outdates

itself).

Caution : Using recycle loops sparingly. Using them without properconsideration may have unexpected side-effects such as downstreamprocesses being executed before upstream processes.

Utility Method Dialog

In process of designing projects you may find that you use similarscript commands repeatedly. Usually they only differ in filenames andfew other arguments. To make the script writing easier and some-whatautomated, rather than writing it from scratch each time, you can useUtilities available to a default installation of Pipe-It to use templatesapplicable to your project. Or create new ones.

See Script Assistant for more details.

8/11/2019 Pipe-It Manaul 20130806

66/264

8/11/2019 Pipe-It Manaul 20130806

67/264


Tool Bar

Application Tool Bar consists of 3 main sections:

Now describe all available actions in the tool bar:

File Operations

This section has 3 main standard file actions as:

Create New Project

Open Project from disk

Save Project

Note that Open Icon has small arrow in right bottom corner. If the

Open icon is pressed and held for one second then a drop-down menuappears with list of recently opened projects.

Navigation, Selection and Zoom

First 3 buttons (greyed out in picture above) allow navigation throughcurrently opened composites. Backand Forwardwork in same way

as navigation buttons in a web browser. Up button changes view toupper-level parent composite. All these buttons can be pressed and
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-View-1.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-FileOpen-1.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-File-1.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-ToolBar-1.png

8/11/2019 Pipe-It Manaul 20130806

68/264


held for one second to get list of history entries or, in case of Up

button, for list of all parents of current composite.

Next 3 buttons Select, Zoom In and Zoom Out manage the current

mouse cursor mode.

Selectallows selection, resizing, moving etc. canvas actions on mouseclick. This is the most common mode of operation.

Zoom Inon click to canvas changes the zoom level for +10%

Zoom Outon click to canvas changes the zoom level for -10%

When Zoom In or Zoom Out is selected, then holding Alt (on Mac:

Option) button temporally switch to opposite zoom action.

Also in Zoom Inor Zoom Out, area selection on canvas switches viewto Zoom-To-Fitthe selection area into view:

Next to it is the, 1:1button. It is not a modebutton, it just switchescurrent view to un-zoomed (1:1) scale or in other words to 100%

scale.

Finally, Last button in the tool bar is Zoom-To-Fit action with drop-down menu. Just pressing the button switch zoom level to fit all yourpresented canvas elements in current view. It has drop-down menu to

apply Zoom-To-Fitto:

Current Window - Ctrl+9

8/11/2019 Pipe-It Manaul 20130806

69/264


Zoom-to-Fit Canvas - Ctrl+6

Elements to insert and Run

The toolbar contains 6 buttons which to add elements to the Canvas.The behavior of these buttons can be summarized as click-and-click.This means the button should be click once on the tool bar and thenthe location on the Canvas should be clicked where that element should

be placed.

Double clicking for continuous insertion mode

You may also double click on buttons on the tool bar. This will turn onthe tool until you choose another one. This will prove practical if youwant to add more than one element of the same type to the canvas.For example if you have lot of connectors to make, just double clickand make all your connectors as you want them to be, without having

to turn the connector tool on for each one.

Insert Resource

This button means adding a Resource to canvas. The resource

object represents a file on disk. Though you can give a several name tothe resource without a connection to actual used file name of the

associated file on disk.

On canvas the object looks like:

First double-click on the object opens Select File Dialogto associate afile with the object. A double-click on the object, that already hasassociation, runs OS defined default Application associated with the file.For example, clicking on a resource with associated "text.txt" file, will

open Notepad.exe

Also you can just drag-and-dropfrom Window Explorer to Canvas a fileobject. It will create a Resource automatically, associate the file toresource object and renames the resource it to File Name of droppedfile.

8/11/2019 Pipe-It Manaul 20130806

70/264


Insert Scripter

This button means adding a Program (Script) to canvas. Thescripter object represents an application to execute.

On canvas the object looks in such way:

Double-click on the object opens Edit Script Dialog.

Sub-menu associated with the button allows to change default action to

insert other elements that are bound to Programcategory.

Insert Manifold, Distributor and Sockets

This button means adding a Manifold to canvas. Manifold gather

information from multiple resources into one visual connector.

Sub-menu associated with the button allows adding other type ofsimilar elements Distributors, Input/Output Resourse/Process

Sockets.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-Manifold-1.png

8/11/2019 Pipe-It Manaul 20130806

71/264


Insert Composite

This button means adding a Composite (Sub-Project) to canvas.

On canvas the object looks in such way:

Look at Composites? and Socket types? pages for more details.

Insert Annotation

The next button allows placement of new connectors between

resources and processes.

Insert Annotation

This button means adding a object with text or image that does notplay role in logic but can represent some illustration, information orinstructions for the project.

Undo Button

provides fast way to revert your changes one step at a time.
http://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolBar/AppHelpToolBar-Undo-1.pnghttp://vmware-host/Shared%20Folders/DATA/Petrostreamz/Development/Distros/BuildManagement/Release_v1.3.0/Docs/contents/ToolB

Documents

Pipe-It Manaul 20130806