EyesWeb 3 Legacy Library Quick Reference

EyesWeb Legacy Blocks

Quick Reference www.eyersweb.org Godbersen 26.10.2005, zusammengetragen + gekürzt + annotiert Sequenz

1. Imaging 2. Sound 3. MIDI 4. Clock 5. Base 6. Serial 7. Math 8. AddOns (aus 3.3) 9. MotionAnalysis 10. BLOB Analysis 11. MoCap

Nicht enthalten: Audio Cues Aktuelle Verfügbarkeit der Blocks mit eywconsole /c prüfen, z.B. stehen in 4.0.11 noch nicht alle Blocks zur Verfügung, bei Bedarf auf 3.3.0 ausweichen.

EyesWeb Imaging Library Quick Reference 1

Imaging Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the Licence provided.

Copyright Notice EyesWeb is Copyright © 1999-2002, Laboratorio di Informatica Musicale - DIST - University of Genoa (http://www.infomus.dist.unige.it)

License agreement Use of EyesWeb (hereinafter 'SOFTWARE') is contingent on your agreement to the following terms: WARRANTY & USE: DIST - University of Genoa grants you a limited,non-exclusive license to use the SOFTWARE free of charge for ANY purpose, commercial or private, without restriction. DIST - Universityof Genoa makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. DIST - University of Genoa is not obligated to provide maintenance or updates for the SOFTWARE. DISTRIBUTION: the SOFTWARE, in original version or in part, may be freely distributed, provided that this copyright and permission notice appear on all copies and supporting documentation, and that the DIST - University of Genoa copyright notices are referred in the following ways:

a) DIST - University of Genoa's copyright notice should be included in the documentation, regardless of the media used to supply the documentation,and

b) The DIST - University of Genoa and EyesWeb Logos have to appear on packages and promotional material (DIST - University of Genoa makes the logos available on the EyesWeb's ftp site ftp://infomus.dist.unige.it), and

c) in the 'about box' of the product, in the case that it is not the EyesWeb about box, DIST – University of Genoa and EyesWeb must be cited in the following manner: "EyesWeb is copyright © Laboratorio di Infomatica Musicale - DIST - University of Genoa (http://www.infomus.dist.unige.it)", and

d) any public use of EyesWeb, or of any application based on EyesWeb, must be preliminarly notified to [email protected].

Antonio Camurri DIST - University of Genoa Viale Causa, 13 16145 Genova, Italy E-mail: [email protected] This work is based in part on the Intel IPL and OpenCV libraries, Copyright © Intel Corporation


Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 1 Conversion.................................................................................................................................................. 4

1.1 BitonalToGray ........................................................................................................................................ 4 1.2 ColorModel............................................................................................................................................. 4 1.3 ColorToGray........................................................................................................................................... 4 1.4 Convert ................................................................................................................................................... 4 1.5 ConvertFromMatrix ................................................................................................................................ 5 1.6 ConvertFromMatrixEx............................................................................................................................ 5 1.7 ConvertToMatrix .................................................................................................................................... 5 1.8 GrayToColor........................................................................................................................................... 5 1.9 HSVToRGB............................................................................................................................................ 5 1.10 ReduceBits.............................................................................................................................................. 6 1.11 ThresholdBitonal .................................................................................................................................... 6

2 Input............................................................................................................................................................ 7 2.1 AviRead .................................................................................................................................................. 7 2.2 AviReadSync .......................................................................................................................................... 7 2.3 BitmapRead ............................................................................................................................................ 8 2.4 FilledFrame............................................................................................................................................. 8 2.5 FrameGrabber ......................................................................................................................................... 9 2.6 FrameGrabberV4W ................................................................................................................................ 9 2.7 MultimediaFileRead ............................................................................................................................. 10

3 Output ....................................................................................................................................................... 11 3.1 AVIWriter............................................................................................................................................. 11 3.2 ClickDisplay ......................................................................................................................................... 11 3.3 Display.................................................................................................................................................. 12 3.4 ManParamsShow .................................................................................................................................. 12

4 Filter ......................................................................................................................................................... 12 4.1 LinearFilterBlur .................................................................................................................................... 12 4.2 LinearFilterFixed .................................................................................................................................. 12 4.3 NonlinearFilter...................................................................................................................................... 12

5 Operations................................................................................................................................................. 13 5.1 AlphaBlend ........................................................................................................................................... 13 5.2 BgndSubMultThresh............................................................................................................................. 13 5.3 ChromaKeyExtractor ............................................................................................................................ 13 5.4 ComposeChannels ................................................................................................................................ 14 5.5 Decimate ............................................................................................................................................... 14 5.6 Deinterlacer .......................................................................................................................................... 14 5.7 DyadicArithmeticOp............................................................................................................................. 14 5.8 DyadicComparison ............................................................................................................................... 14 5.9 DyadicLogicalOp.................................................................................................................................. 15 5.10 ExcludeAreas........................................................................................................................................ 15 5.11 Extract................................................................................................................................................... 15 5.12 ExtractChannel ..................................................................................................................................... 15 5.13 ExtractColor.......................................................................................................................................... 15 5.14 ExtractMultColors ................................................................................................................................ 16 5.15 FlatInfiniteIntegrator............................................................................................................................. 16 5.16 ImageDescription.................................................................................................................................. 16 5.17 ImageIntegrator..................................................................................................................................... 17 5.18 ImageSplitter ........................................................................................................................................ 17 5.19 InfiniteIntegrator................................................................................................................................... 18 5.20 Interlacer ............................................................................................................................................... 18 5.21 Merge.................................................................................................................................................... 18 5.22 MergeOffset.......................................................................................................................................... 19 5.23 Mirror.................................................................................................................................................... 19 5.24 MonadicArithmeticOp .......................................................................................................................... 19


5.25 MonadicLogicalOp ............................................................................................................................... 19 5.26 MorphMirror......................................................................................................................................... 20 5.27 MorphologicalOp.................................................................................................................................. 20 5.28 MoveImg .............................................................................................................................................. 20 5.29 Resize.................................................................................................................................................... 20 5.30 Rotate.................................................................................................................................................... 20 5.31 SimpleSumOfDifferences..................................................................................................................... 20 5.32 SimpleSumWReset ............................................................................................................................... 21 5.33 Threshold .............................................................................................................................................. 21 5.34 ThresholdBitonal .................................................................................................................................. 21 5.35 WeightedSumOfDifferences................................................................................................................. 22 5.36 VideoMixer........................................................................................................................................... 23 5.37 Zoom..................................................................................................................................................... 23

6 FeatureCalc............................................................................................................................................... 23 6.1 Area ...................................................................................................................................................... 23 6.2 Barycenter............................................................................................................................................. 23 6.3 BoundingRect ....................................................................................................................................... 23 6.4 CannyCornerDetection ......................................................................................................................... 23 6.5 CentroidsCalc ....................................................................................................................................... 24 6.6 ManParamsCalcExtract ........................................................................................................................ 25 6.7 Moment................................................................................................................................................. 25 6.8 Moments ............................................................................................................................................... 25 6.9 OpticalFlowLk...................................................................................................................................... 26 6.10 OpticalFlowPyrLk ................................................................................................................................ 26

7 Draw ......................................................................................................................................................... 26 7.1 2PointElement....................................................................................................................................... 26 7.2 DrawPoint ............................................................................................................................................. 26 7.3 ManParams ........................................................................................................................................... 27


Imaging Library The Imaging Library is contained in the EywImaging.dll, EywImagingDatatype.dll and EywImagingExtras.dll DLL files. It contains blocks for image processing and handling, based in part on the Intel IPL and OpenCV Libraries.

1 Conversion

1.1 BitonalToGray

Parameters

Destination depth depth of the resulting image Zero scale resulting color value for input 0s One scale resulting color value for input 1s

Remarks

Converts a two-color (1 bit per pixel) image to a normal monochrome (grayscale, >1 bpp) image. The output color resolution may be chosen setting the Destination depth parameter, while the Zero scale and One scale parameters allow to specify which color values should correspond in the output image to the 0s and 1s in the input image.

1.2 ColorModel Parameters

Destination color model color model of the resulting image

Remarks

Converts among the different color formats supported by the IPL library. The Destination color model parameter specifies the output image format (IPL Color Model).

1.3 ColorToGray Parameters

Depth depth of the resulting image

Remarks

Converts a color (RGB, BRG) input image to a monochrome image. The Depth parameter allows to change the output image depth.

1.4 Convert Parameters

Destination depth depth of the resulting image Destination data ordering

data ordering of the resulting image

Destination origin origin of the resulting image

Remarks

Converts any image type to the format specified by Destination depth, Destination data ordering and Destination origin. All properties can be changed simultaneously.


1.5 ConvertFromMatrix Parameters

Depth depth of the resulting image Order data ordering of the resulting image Origin origin of the resulting image

Remarks

Reads from input a matrix and interprets its elements as image pixel values; the resulting image is provided as output. The format of the output image should be specified using the Depth, Order and Origin parameters.

1.6 ConvertFromMatrixEx Remarks

Reads from input a matrix and interprets its elements as image pixel values; the resulting image is provided as output. The format of the output image is the same as the image provided as the second input (and therefore the Depth, Order and Origin parameters are not needed). Note that the second input is used only during the Init phase to determine the output image format, after which it is ignored.

1.7 ConvertToMatrix Remarks

Converts the input image to a matrix. The matrix has the same dimensions of the input image, which, in turn, should be monochrome.

1.8 GrayToColor Parameters

FractR Red plane scale of the resulting image FractG Green plane scale of the resulting image FractB Blue plane scale of the resulting image

Remarks

Converts a monochrome input image to a color image. The conversion is performed creating a three color plane image, in which the planes are equal, minus a scale factor. The FractR, FractG and FractB parameters specify the corresponding color plane scale, between 0.0 and 1.0.

1.9 HSVToRGB

CLSID: 0xE779190C,0x9A3D,0x4980,{0x88,0x68,0x2F,0x1B,0x4A,0x38,0x8C,0x9B}

Inputs

H (Main) MathDatatype.Scalar The H value of a color expressed in HSV coordinates. H must be a real number in the range [0, 360].

S MathDatatype.Scalar The S value of a color expressed in HSV coordinates. S must be a real number in the range [0, 1].

V MathDatatype.Scalar The V value of a color expressed in HSV coordinates. V must be a real number in the range [0, 1].

Outputs

R MathDatatype.Scalar The R value of a color expressed in RGB coordinates. R is


a real number in the range [0, 1]. G MathDatatype.Scalar The G value of a color expressed in RGB coordinates. G

is a real number in the range [0, 1]. B MathDatatype.Scalar The B value of a color expressed in RGB coordinates. B is

a real number in the range [0, 1]. Remarks

Converts a color expressed in the HSV color model in the corresponding color expressed in the RGB color model. Note that the incoming and outgoing values are scalar numbers, that is, this block does not operate on the whole pixels of an image, but on a single pixel value. Error and warning messages

During the init phase Message Description

An error occurred while getting the init data. The block could not obtain the init data of some of its inputs.

H must be real. H must be a real number in the range [0, 360]. S must be real. S must be a real number in the range [0, 1]. V must be real. V must be a real number in the range [0, 1]. During the execution phase Message Description

An error occurred in the locking phase. The block could not lock an input or an output. The execution phase will fail.

1.10 ReduceBits Parameters

Destination depth depth of the resulting image Levels number of levels in the resulting image Noise noise value Dithering type type of dithering to apply to the resulting image

Remarks

Decreases the number of levels used in the input image. It is also possible to change the output image depth using the Destination depth parameter. The Levels parameter specifies the number of levels to be used in the output image, and thus depends on the image depth (e.g. at most 256 levels can be specified for an 8-bit image). If Noise is set to a value greater than zero, a random noise is added to the threshold levels used for computation. Dithering is also allowed, specifying the Dithering type parameter.

1.11 ThresholdBitonal Parameters

Threshold integer threshold value

Remarks

Converts a monochrome input image to a two-color (1 bpp) image. If a pixel in the input image exceeds the value specified for the Threshold parameter, the corresponding pixel in the output image is set to 1, otherwise it is set to 0.


2 Input

2.1 AviRead Remarks

Reads a video stream from an AVI file. The file can be selected using the AviFileName parameter: the choice, either entered directly or via the Browse button, should be applied to be effective. To set all base parameters from the file properties (FrameRate, Start/Stop frame) use the SetOrigParms command. Note that playback does not start automatically when the patch is started, but the Play command must be issued (such a command may nevertheless be linked to the starting of the patch using a paramchanger). Since version 2.4 AviRead is superseded by the MultimediaFileRead block, that support a broader range of supported file types. Parameters

AviFileName AVI file name and path FrameRate video frame rate (in frames per second) StartFrame starting video frame StopFrame stop video frame

2.2 AviReadSync CLSID: 0x845B224D,0x9B9E,0x48F2,{0xA8,0xD3,0xCF,0x8F,0xF3,0x00,0xD3,0x0B}

Inputs

Input clock Any Outputs

Output image Imaging.Image Parameters

AviFileName Filename Frame Rate Integer Start Frame Integer Stop Frame Integer SetOrigParams Command Continuous Output Boolean Loop Boolean Play, Pause, Stop Commands

Remarks

Reads an AVI file and produces a single frame as output. The frame rate depends on the speed of the input signal, i.e., a new frame is produced each time a signal is received as input. Since the input is used only for implementing this activation mechanism, its type is not important (and its value is not used); thus, any type of data handled by EyesWeb can be used as input. The AviFileName parameter determines the file to be used; only AVI files are supported, and the type of compressions allowed depends on the video codecs that are installed on the computer. The Start and Stop Frame parameters determine the subset of the file to be used, i.e., they may be used to skip the beginning or the end (or both) of the file. If they are both zero, then the whole file is used. To know the maximum allowed value for Stop Frame, click on the SetOrigParams button once an existing file has been selected. The Continuous output parameter controls the behaviour of the block when it is in paused or stopped mode: if it is checked, then the last used frame is continuously sent out (whenever an input signal is received), otherwise, no output is generated and the depending blocks will stop processing data. Finally, the Play, Pause, and Stop commands have the usual meaning. Note that when the patch is started, the block is initially stopped. To make it automatically start, you can, for example, connect an appropriately configured param changer to its exported Play command, as in the following example:


Note that the main difference with the now unsupported AviRead block lies in the fact that the clock of this block is provided through the input.

2.3 BitmapRead Parameters

BitmapPath BMP file name and path Refresh mode Manual | Periodic BMP file name and path

Remarks

Reads a bitmap image from file. The file can be chosen with the BitmapPath parameter, either entering its value directly or via the Browse button.The Refresh mode parameter controls how output is generated: when set to Manual the image is output upon receiving the Refresh command, otherwise it is output periodically with a period (in Hz) specified by the FrameRate parameter. The Reload command allows to re-read the image from file in case it has changed, since it is normally read only during the Init phase and kept in an internal buffer during execution.

2.4 FilledFrame Parameters

BitmapPath BMP file name and path Refresh mode Manual | Periodic BMP file name and path

Remarks

Generates a uniform color image. The available parameters can be used to set the dimensions in pixels, color or B/W mode and color components (RGB for Color or Gray for B/W). The color components’ value should be an integer between 0 and 255. The output image is generated periodically with a period (in Hz) specified by the FrameRate parameter. The PickColor command allows the color selection via the standard Windows Color Picker Common Dialog.


2.5 FrameGrabber CLSID: 0x26A009CB,0x2437,0x4C7B,{0x94,0x88,0x18,0x9B,0xD0,0x1E,0x1B,0x57}

Outputs

Output image Imaging.Image

Parameters

Device Combo Grabber Properties Command Capture Properties Command Type Combo Width Integer Heigth Integer Fps Integer

Remarks

Captures images from any type of video input source, provided that its drivers are fully WDM (Windows Driver Model) compliant and correctly installed. Most frame grabbers, TV tuners, and WebCams now come with WDM support and can be used as video sources in EyesWeb through this block. Note that the FrameGrabberV4W block is functionally very similar but only supports sources using Video for Windows drivers. The available devices are listed in the Device combo; once the device has been selected, the capture properties must be set through the Grabber properties and Capture properties commands. When one of these commands is selected, a dialog appears to modify some settings of the frame grabber. Note that this dialog is provided by the device driver, hence, they can differ from one device to the other, and from one system to another.

The following figure, for example, shows the dialog that appears on some systems, when Grabber Properties is selected for a Hauppauge WinTV GO TV tuner card. From this dialog it is possible to choose the Video Source and the Video Format to use when capturing. The Video Format command, in this case, lets you choose the dimensions of the output image of the block. Note that this setting can usually be provided directly from within the EyesWeb param dialog of the block. However, it is possible that the settings used in EyesWeb for the Width, Height, and Type parameters are not supported by some devices. Hence, the possibility to choose these values directly through a

driver-provided dialog box can at times be particularly useful. Finally, the Fps (Frames per second) option lets you to select the capturing frame rate.

2.6 FrameGrabberV4W Parameters

Device index Video acquisition device ID Frame width Video frame width Frame height Video frame height


Mode Video mode

Remarks Acquires a video stream from a video input device that fully supports the Video for Windows standard. The Device index parameter is used to select which device to use when more than one are present. Note that in some cases more entries are present even if only one physical device is installed, e.g. when different releases of the driver are installed. The size and type (color or B/W) of the image can be set using the FrameWidth, FrameHeight and Mode parameters. Not all settings may be supported by all devices; in this case it is necessary to use the device driver dialogs to set a supported format: this can be done choosing Use video format dialog for the Mode parameter. The VideoFormat command then allows to choose directly from a device supported list (a 24-bit RGB is recommended if available). Currently, this block supports the color and B/W formats (RGB24 and RGB8, respectively); YUV9 and YUV12 formats are also supported, but they can only be chosen by means of the VideoFormat dialog (put the Mode combo to Use format dialog to enable the Video format command). USB WebCams may require particular tips to work properly. If you cannot use a USB grabber, by simply choosing the format using the block parameters, try the following steps. Prepare a patch with the FrameGrabberV4W block, setting the Mode parameter of the FrameGrabberV4W block to Use format dialog, then save and close the patch. Open the patch again and Open the Video_Format dialog (by clicking on the button in the FrameGrabberV4W dialog). A dialog should appear, where you can modify the resolution and the color format (choose, for instance, RGB24, 384x288). Click OK to close this dialog. It may happen that EyesWeb crashes just when you choose OK. Anyway, the format should have been stored, so when you open the patch again, you do not have to change the format again. Simply start the patch execution. If the format dialog didn’t appear (that is, EyesWeb crashed when clicking on the Video_Format button) the device index (the first parameter of the FrameGrabberV4W block) may be wrong: specify another value (0 to 9 are allowed), then modify the video format as described above. Note that, after changing the Device index value, you have to choose Apply before opening the Video format dialog, otherwise you keep working with the old device index. Moreover, note that when the patch is locked, the name of the device found at the specified index, if any, is written in the EyesWeb message bar, e.g.: (Message) Capture successfully initialized. Device: Logitech USB Video Camera; Version: 5.4.3.1017 If neither of these solution works, a last resort is to modify the video format through one external tool (Microsoft VidCap, for instance), often supplied together with the frame grabber. So, perform the first step, i.e. set the mode to Use format dialog and save the patch; now, find a tool that lets you modify the color format and resolution, and use RGB24 (or another supported format, that is, RGB8, YUV9, YUV12) as the color format. Use the frame grabber for a while in this modality (this ensures that the desired format is permanently stored). Close the external tool and try to start the patch.

2.7 MultimediaFileRead CLSID: 0x2F364A53,0xC660,0x4CBF,{0x90,0x3E,0x1F,0xED,0x44,0x72,0x03,F1}

Outputs

Output image Imaging.Image

Parameters

AviFileName Filename Frame Rate Integer Start Frame Integer Stop Frame Integer SetOrigParams Command Continuous Output Boolean Loop Boolean Play, Pause, Stop Commands

Remarks

Reads AVI or MPEG files, and produces the single frame as output. Differently from the AviReadSync block, the timing of this block is not determined by an input signal, but from the timeing information contained in the file itself. Another difference is in the type of compatible file formats: this block, besides AVI files, supports MPEG compressed files and many other file formats.


The AviFileName parameter determines the file to be used; The Start and Stop Frame parameters determine the subset of the file to be used, i.e., they may be used to skip the beginning or the end (or both) of the file. If they are both zero, then the whole file is used. To know the maximum allowed value for Stop Frame, click on the SetOrigParams button once an existing file has been selected. The continuous output parameter controls the behaviour of the block when it is in paused or stopped mode; if it is checked, then the last used frame is continuously sent out (whenever an input signal is received), otherwise, no output is generated and the depending blocks will stop processing data. Finally, the Play, Pause, and Stop commands have the usual meaning. Note that both the AviReadSync and the MultimediaFileRead blocks can be managed through their custom dialog, that provides a more user friendly interface:

3 Output

3.1 AVIWriter Remarks

Writes the incoming stream of images to an AVI file. The supported formats are B/W, in uncompressed mode, or 24bit color, in compressed or uncompressed mode. The type of compression can be selected through the Video Compression command.

3.2 ClickDisplay Remarks

Same as the Output.Display block, except its ability to output (as a 1x2 matrix) the position of a mouse click on the displayed image.


3.3 Display Remarks

Displays the image in a moveable, resizeable and minimizeable window. The StretchInterpolation parameter indicates which interpolation algorithm to use when the window and image sizes differ: from the least-CPU- intensive to the most-CPU-intensive are available Nearest neighbor, Linear and Cubic (which gives the best results). The MinimizeWithGUI parameter, when checked, indicates that the display window is to be minimized with the EyesWeb Application; the window remains visible otherwise. Right-clicking on the window displays a context menu that allows to choose among several predefined zoom values. Note that the scheduler is paused (stopped) while this menu is visible and does not restart until the menu hides. This behavior is by design and may change in future releases.

3.4 ManParamsShow Remarks

Opens a window where a stylized human silhouette is drawn with the superimposed positions of the main point of a human body. The positions, together with other info such as the occupied area in pixels, the bounding rectangle width and height, are contained in the row vector that has to be provided as input to this block. Note that the format of the vector as expected by this block, is the same as the vector generated by the FeatureCalc.CentroidsCalc block.

4 Filter

4.1 LinearFilterBlur Parameters

NumberOfColumns number of matrix columns NumberOfRows number of matrix rows Enabled true | false enable/disable filter

Remarks

Blurs the image. The NumberOfColumns and NumberOfRows parameters indicate the size of the matrix used in the algorithm. The higher their value, the greater the effect but also the CPU consumption. The Enabled parameter, when unchecked, cancels the effect, actually bypassing the block.

4.2 LinearFilterFixed Parameters

Type filter type (IPL)

Remarks

Applies a linear filter among those provided by the IPL library. The filter type is selected using the Type parameter; the Nop (NoOperation) setting indicates that the image remains unaffected. The meaning of the filters (i.e. the convolution matrix applied) is documented in par. 6.12 of the IPL 2.5 manual. Note that several linear filters chained together can be permuted without affecting their overall effect; the same applies even if multiplications by constant (MonadicArithmeticOp) are present.

4.3 NonlinearFilter


Parameters

NumberOfColumns number of matrix columns NumberOfRows number of matrix rows Type filter type

Remarks

Applies a nonlinear filter using the FilterType parameter; the Nop (NoOperation) setting indicates that the image remains unaffected. The NumberOfColumns and NumberOfRows parameters indicate the size of the matrix used in the algorithm. For further information please refer to par. 6-14 of the IPL 2.5 manual.

5 Operations

5.1 AlphaBlend Inputs

Foreground Imaging.Image The image that will be used as foreground Mask Imaging.Image The mask specifying the alpha values Background Imaging.Image The image that will be used as background

Parameters

Blending Mode Specifies the blending technique to be used. Please refer to the IPL manual for details.

X Shift X-axis offset between foreground and mask images Y Shift Y-axis offset between foreground and mask images

Remarks

The AlphaBlend block performs alpha blending of two images. The mask (an image) passed as a parameter specifies the alpha values. Foreground and Mask must have the same width and height, while Background may be larger than the other two.

5.2 BgndSubMultThresh Remarks

Subtracts the background from an image. See the sample EyesWeb patches Silhouette extraction and Silhouette extraction-enhanced.

5.3 ChromaKeyExtractor Parameters

U Channel Threshold integer Threshold on the U channel V Channel Threshold integer Threshold on the V channel Invert U Channel true | false

Enables/disables inverting the U channel (after applying the threshold)

Invert V Channel true | false Enables/disables inverting the V channel (after applying the threshold)

Image Type Input image type

Remarks

Extracts blobs from the input image using a Chroma Key process and outputs the resulting image. The input image should be in YUV format (YUV9 or YUV12). Sample color U V BInv U BInv V

Blue 190 0 False False


Red 0 255 False False Yellow 40 130 True False Green 60 50 True True Magenta 170 140 False False Cyan 140 70 False True

5.4 ComposeChannels Parameters

Color model Color model

Remarks

Outputs a three-color image given three monochrome (single-channel) images as input. The meaning of each channel depends on the Color model parameter, e.g. a value of BGR will have the block interpret the first channel as (B)lue, the second as (G)reen and the third as (R)ed.

5.5 Decimate Parameters

X Scale factor X-axis scale factor Y Scale factor Y-axis scale factor

Remarks

Decreases the size of the image by the factors specified in the X Scale factor and Y Scale factor parameters, which should be equal to or greater than 1. The higher their value, the greater the decrease in size (the smaller the resulting image). Note that size can only be reduced or left unaltered, not increased.

5.6 Deinterlacer Remarks

Deinterlaces the image, separating even and odd lines. The two resulting images have the same width and half the height of the input image, the first being made up from lines 1, 3, 5, 7, … of the original and the second from lines 2, 4, 6, 8, …

5.7 DyadicArithmeticOp Parameters

Operation Type type of arithmetic operation

Remarks

Performs arithmetic operations on two images (binary/dyadic operations). The type of the operation is selected with the Operation Type parameter, No operation meaning that the first input is output directly without modification. Add, Subtract and Multiply operate between the corresponding pixels of the two images (that should both have the same properties, i.e. dimensions, number of channels and so forth). Subtract inverse works like Subtract but inverting the source order (i.e. B - A instead of A - B). Subtract Absolute computes the absolute value of the difference. Multiply Scale performs multiplication ensuring no saturation affects the result (e.g. 255 x 255 = 255).

5.8 DyadicComparison


5.9 DyadicLogicalOp Parameters

Operation Type type of logical operation

Remarks

Performs logical operations on two images (binary/dyadic operations). The type of the operation is selected with the Operation Type parameter, No operation meaning that the first input is output directly without modification. All operations are performed bitwise, i.e. between corresponding pixels.

5.10 ExcludeAreas Remarks

Removes some rectangular zones from the input image, by setting these zones to a specified color. The zones to be excluded are selected with an external block (e.g., the ClickDisplay block). More than one zone can be managed by the block. The block has three inputs: the first is the source image, i.e., the image where some parts must be removed; the second is an image that should be the same size (width, height) as the first one; the third is a matrix containing the coordinates of a rectangle (the ClickDisplay block, when set in rectArea mode, generates a compatible matrix as output). The two outputs of the block are the source image with the excluded areas, and a second image where the removed rectangle are depicted. The custom dialog of the block lets you manage the rectangles, by choosing if the incoming one is a new rectangle, or is to be considered as a modification to one of the existing one(s).

5.11 Extract Parameters

Width width of the subimage Height height of the subimage X X-axis position of the top left corner of the subimage Y Y-axis position of the top left corner of the subimage Remarks

Extracts a subimage from the image. The Width and Height parameters specify the dimensions of the output image, while the X and Y specify the position with respect to the input image.

5.12 ExtractChannel Parameters

Channel channel ID

Remarks

Extracts a channel from the input image. Channels are specified with the Channel parameter starting from 1, e.g. extracting channel 1 from a three-channel RGB image returns a monochrome image made up from the single red image component.

5.13 ExtractColor Parameters

OutImageDepth depth of the resulting image (RGB or grayscale)


GetColorMode when unchecked, any click in the connected ClickDisplay is interpreted as an override for the COG coord. of the selected blob (in the custom dialog). When checked, the search color of the selected blob is also changed to the color of the clicked pixel

Remarks

Extracts a color blob from an input RGB image, calculating the blob baricenter.. The output is the image rendered according to the result of the analysis.

5.14 ExtractMultColors Parameters

OutImageDepth depth of the resulting image (RGB or grayscale) GetColorMode when unchecked, any click in the connected ClickDisplay

is interpreted as an override for the COG coord. of the selected blob (in the custom dialog). When checked, the search color of the selected blob is also changed to the color of the clicked pixel.

DrawGraph when checked, a graph of the cog, bounding rect and search rect is overlayed on the output image

Remarks

Extracts multiple color blobs from an input RGB image, calculating the blob baricenters, areas and bounding rectangles. HSV thresholding is performed, using specified threshold levels and search rectangle dimensions. The search rectangle is used to restrict the blob search process to a limited area, thus reducing CPU usage. The inputs are the image to be processed, which should be in BGR format, and a 1x2 matrix representing a 2D point (this input must be connected to a DisplayClick block). The outputs are the image rendered according to the result of the analysis; a Nx2 matrix, where N is the number of the blobs, holding in each row the (x,y) coordinate of the corresponding blob baricenter; a Nx1 matrix, where N is the number of the blobs, holding in each row the area of the corresponding blob baricenter; a Nx4 matrix, where N is the number of the blobs, holding in each row the (x1,y1)-(x2,y2) coordinates of the corresponding blob bounding rectangle.

5.15 FlatInfiniteIntegrator Parameters

Reset Clears the internal accumulator

Remarks

Implements a continuos OR of images, with the possibility to reset (clear) the result. It actually ORs every incoming image to an internal accumulator that can be cleared with the Reset parameter.

5.16 ImageDescription Parameters

Field queried property

Remarks

Outputs the description of the input image property specified by the Field parameter. Width returns the width, in pixels, of the image; Height returns the height, in pixels, of the image; Depth returns the depth of each color channel (see below); Number of channels returns the number of color channels of the image; Color model returns the color model of the image (see below); Alpha channel returns 1 if the input image has an alpha channel, 0 otherwise; Flipped returns 1 if the image is flipped, 0 otherwise; Pixel ordered returns 1 if the image


is in pixel order (RGBRGBRGB...), 0 otherwise (RRR...GGG...BBB...); Quad word aligned returns 1 if the image is aligned on a quad word, 0 otherwise. Depth values

1 single bit 8 8 bit unsigned

16 16 bit unsigned 32 32 bit float

-2147483640 8 bit signed -2147483632 16 bit signed -2147483616 32 bit signed

Color Model values

0 BGR 1 RGB 2 GRAY 3 HSV 4 HLS 5 XYZ 6 YUV 7 Ycr 8 YCC 9 LUV

10 Undefined

5.17 ImageIntegrator Parameters

N Number of images to add.

Remarks

Computes the sum of N images (where N is a parameter of the block) and outputs their average in one of two possible depths: IPL_DEPTH_8U (that is unsigned byte) or IPL_DEPTH_16S (that is signed short integer).

5.18 ImageSplitter CLSID: 0x7A496756,0xB6DE,0x4EC5,{0x9C,0x1A,0x35,0xAD,0xD5,0x60,0x5C,0xED} Inputs

Input Image (Main) Imaging.Image The image to split. Coords MathDatatype.Matrix A vector containing the x and y coordinates of the 2D

point in correspondence of which splitting is performed. It must be a 1x2 vector of integers. Note that when splitting on “Bottom-Up” mode, the y coordinate of the 2D point should be referred to a top left origin.

Outputs

Right/Top Half

Imaging.Image The image containing the right part of the input image when splitting in “Left Right” mode or the top part of the input image when splitting in “Bottom Top” mode.

Left/Bottom Half

Imaging.Image The image containing the left part of the input image when splitting in “Left Right” mode or the bottom part of the input image when splitting in “Bottom Top” mode.


Parameters

Mode Combo This parameter is used to indicate if the input image should be split in its left and right parts or in its bottom and top parts. The default value is “Left - Right splitting”.

Remarks

Splits an image into left and right or bottom and top parts according to the x or y coordinate of a 2D point provided as input. Note that the output images have the same dimensions of the input image: the remaining half is filled in black. Notice also that when splitting in “Bottom-Top” mode, the y coordinate of the 2D point should be referred to a top left origin. Error and warning messages

During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. The input vector must be 2 columns x 1 row (2D point).

The input 2D point must be expressed as a vector having one row and two columns. Check the dimensions of your input vector.

The input vector must be an integer 2D point. The coordinates of the input 2D point must be integer numbers. You can either use a vector of integers or a “DomainConv” block the EyesWeb Math.Matrix library to convert a vector of floats in a vector of integers.

During the execution phase Message Description An error occurred in the locking phase. The block could not lock an input or an output. The

execution phase will fail.

5.19 InfiniteIntegrator Parameters

Reset Resets the result

Remarks

Adds together all of the images it receives and outputs the result scaled by the number of processed images. The Reset command allows to reset the result.

5.20 Interlacer Remarks

Interlaces the two input images into one. The resulting image has the same width but twice the height of the input images; the lines of the first image make up the odd lines (1, 3, 5, 7, …), while those of the second make up the even lines (2, 4, 6, 8) of the resulting image.

5.21 Merge Parameters

Mode merge type


Remarks

Tiles two images, horizontally or vertically depending on the Mode parameter.

5.22 MergeOffset Parameters

X X-axis position of the top left corner of the second image Y Y-axis position of the top left corner of the second image

Remarks

Inserts the image from the second input into the image from the first input. The output image has the same size of the first input image. The X and Y parameters specify (in pixels) the position where the second image should be inserted with respect to the first image.

5.23 Mirror Parameters

Horizontal Axis true | false enable/disable horizontal mirror Vertical Axis true | false enable/disable vertical mirror Remarks

Mirrors an image on the horizontal or vertical axis (or both). The Horizontal Axis and Vertical Axis parameters specify which mirror should be applied; if both are left unchecked the output image is unaffected.

5.24 MonadicArithmeticOp Parameters

Operation Type type of arithmetic operation Value second operand

Remarks

Performs arithmetic operations on one image (unary/monadic operations). The operation is selected with the Operation Type parameter, No operation meaning that the input is output directly without modification. The Value parameter must be specified where the selected operation requires it. Add, Subtract and Multiply use this parameter as the second operand, adding, subtracting or multiplying by its value each pixel in the image. Multiply Scale ensures that the result of the operation remains within the image range, e.g. a pixel with a value of 255 multiplied by 255 returns 255. Subtract inverse works like Subtract but inverting the source order (B - A instead of A - B). Square returns an image where each pixel is the square root of the pixel in the original image, ignoring the Value parameter.

5.25 MonadicLogicalOp Parameters

Operation Type type of arithmetic operation Value second operand

Remarks

Performs logical operations on one image (unary/monadic operations). The operation is selected with the Operation Type parameter, No operation meaning that the first input is output directly without modification. All operations are performed bitwise, i.e. between corresponding pixels. And, Or, Xor operate between the image and the Value parameter; Left shift and Right shift perform a shift of Value bits. Not inverts the input bits, ignoring the Value parameter.


5.26 MorphMirror

5.27 MorphologicalOp Parameters

Operation Type type of logical operation Num iterations number of iterations

Remarks

Performs IPL morphological operations on the image. The operation is selected with the Operation Type parameter; the Num iterations parameter specifies how many iterations should be applied. For further information please refer to par. 8 of the IPL 2.5 manual.

5.28 MoveImg Parameters

Xshift Integer X-axis offset Yshift Integer Y-axis offset

Remarks

This block moves the image of xshift pixels to the bottom and yshift pixels to the right. Only positive values are allowed.

5.29 Resize Remarks

Combines the Decimate and Zoom capabilities, allowing also mixed operations, e.g. to reduce one dimension and increase the other. Different blocks are provided since the specialized ones may prove more performing.

5.30 Rotate Parameters

Angle rotation angle in degrees X X-axis position of center Y Y-axis position of center Interpolation type of interpolation Smooth edges enable/disable smoothing

Remarks

Rotates the image by the amount in degrees specified by the Angle parameter, using as a center the point specified by the X and Y parameters (in pixels, can be positioned outside the image). The Interpolation parameter allows to select the interpolation type and the Smooth edges parameter applies a smoothing effect to the image borders.

5.31 SimpleSumOfDifferences CLSID: 0x6E1BF75B,0x6F33,0x4C35,{0xA6,0x97,0x94,0xF8,0x07,0x2A,0x6A,0xB4} Inputs

BW input image Imaging.Image The current input frame. It must be a single channel image.


Outputs

Output image Imaging.Image The output image. It is the sum of the desired number of differences between the current and previous frame.

Parameters

Number of differences Integer [2, +�� [Design Time] The number of differences between the current and previous frame that will be summed together to obtain the output image. The default value is 4.

Remarks

Computes the sum of N image differences, where N is set as parameter (the “Number of differences” parameter). Each time a new frame arrives, the difference with the previous one is calculated. Then, this difference is summed up with the previously calculated N – 1 differences. The resulting image is returned as output. Error and warning messages

During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. Just single channel images allowed. The input image must be a single channel image. Check

your input image. Eventually, you can convert it in a single channel image by using the “ColorToGray” block in the EyesWeb Imaging.Conversion library



5.32 SimpleSumWReset Parameters

Reset Clears the internal accumulator.

Remarks

Computes the sum of N images, using a simple addition operation (iplAdd). No scaling is performed on the result. The reset parameter is a command that allows the internal accumulator to be cleared.

5.33 Threshold Parameters

Threshold threshold value

Remarks

Performs a threshold operation on a monochrome image, in which the pixels of the resulting image are set to the extreme values in the image range (e.g. 0 and 255 for an 8-bit image) depending on the original value being lower or higher than the level specified by the Threshold parameter.

5.34 ThresholdBitonal


Parameters

Threshold threshold value

Remarks

Performs a threshold operation on a monochrome image, in which the pixels of the resulting image are set to 0 or 1 (generating a two-color image) depending on the original value being lower or higher than the level specified by the Threshold parameter.

5.35 WeightedSumOfDifferences CLSID: 0xF3B3745A,0xA9EC,0x45EC,{0x85,0xCB,0xDF,0x5D,0x24,0xFD,0xF1,0xE6} Inputs

BW input image (Main) Imaging.Image The current input frame. It must be a single channel image.

Weights MathDatatype.Matrix A vector containing the weights that have to be applied. It must be a vector of integers in the range [0, 255].

Outputs

Output image Imaging.Image The output image. It is the weighted sum of the desired number of differences between the current and previous frame.

Parameters

This block does not have any parameter. The desired number of differences is given by the number of elements in the Weights input vector.

Remarks

Computes the weighted sum of N image differences, where N is given by the number of element in the Weights input vector. Each time a new frame arrives the difference with the previous one is calculated. Then, the weighted sum of this difference with the previous N – 1 differences is calculated, according to the weight vector provided as input. The resulting image is returned as output. Error and warning messages


inputs. Just single channel images allowed. The input image must be a single channel image. Check

your input image. Eventually, you can convert it in a single channel image by using the “ColorToGray” block in the EyesWeb Imaging.Conversion library

The input vector must be vector of integers.

The weights must be integer numbers between 0 and 255. You can either use a vector of integers or a “DomainConv” block in the EyesWeb Math.Matrix library to convert a vector of floats in a vector of integers.




5.36 VideoMixer Parameters

Mix Real [0..1]

Remarks

Mixes two images together. The resulting image is computed as

OutputImage = Mix * Image1 + ( 1 – Mix ) * Image2

5.37 Zoom Parameters

X Scale factor X-axis zoom factor Y Scale factor Y-axis zoom factor

Remarks

Increases the size of the image by the factors specified in the X Scale factor and Y Scale factor parameters, which should be equal to or greater than 1. The higher their value, the greater the increase in size (the bigger the resulting image).

6 FeatureCalc

6.1 Area Remarks

Counts the number of non-zero pixels in the input image. If no such pixels are found, returns Invalid.

6.2 Barycenter Remarks

Computes the image barycenter and outputs its position as a 1x2 matrix containing the X and Y coordinates. The computation is performed according to par. 12-5 of the IPL 2.5 manual.

6.3 BoundingRect Remarks

Computes the bounding rectangle size and outputs a 2x2 matrix containg the X and Y coordinates of its top left and bottom right corners. The bounding rectangle is defined as the smallest rectangle that still contains all non-zero pixels in an image.

6.4 CannyCornerDetection Remarks

Outputs an image containing the edges of the input image. For further information please refer to the cvCanny method of the OpenCV library.


6.5 CentroidsCalc Remarks

Computes the position in pixels of some centroids (i.e., barycenters) of some zones of the input blob, assuming that the input image contains an unique blob (connected component). Moreover, the input image must be a monochrome image, with a depth of eight or one bit. It first computes its bounding rectangle, the area (in pixels) that it occupies, and its barycenter. Then, it computes the centroids of other subparts of the blob, whose positions are obtained relatively to the position of the barycenter of the whole blob. The following picture gives a sketch of the iterations performed by the algorithm.

The computed points are a rough approximations of the barycenters of the projection of some points (head, shoulders, hands, elbows, knees, and feet) of a human silhouette on a 2D plane, provided that some conditions on the posture of the human being are satisfied: - the projection of the arms, legs and body must not overlap; - the projection of the arms must be external to the projection of the body (i.e., the arms are assumed to be

between the shoulder and the external sides of the bounding rectangle: an arm kept over the head, for example, fails to satisfy this condition);

- the legs must be under the barycenter of the whole blob. If all the above conditions are satisfied, then the output of the block is a row vector containing the following data: 1. Area occupied by the blob (in pixels) 2. Top-left point of the bounding rectangle, x coordinate 3. Top-left point of the bounding rectangle, y coordinate 4. Bottom-right point of the bounding rectangle, x coordinate 5. Bottom-right point of the bounding rectangle, y coordinate 6. Baricenter, x coordinate 7. Baricenter, y coordinate 8. Head position, x coordinate 9. Head position, y coordinate 10. Shoulder left position, x coordinate 11. Shoulder left position, y coordinate


12. Shoulder right position, x coordinate 13. Shoulder right position, y coordinate 14. Elbow left position, x coordinate 15. Elbow left position, y coordinate 16. Elbow right position, x coordinate 17. Elbow right position, y coordinate 18. Hand left position, x coordinate 19. Hand left position, y coordinate 20. Hand right position, x coordinate 21. Hand right position, y coordinate 22. Knee left position, x coordinate 23. Knee left position, y coordinate 24. Knee right position, x coordinate 25. Knee right position, y coordinate 26. Foot left position, x coordinate 27. Foot left position, y coordinate 28. Foot right position, x coordinate 29. Foot right position, y coordinate Note that all positions are given in pixels, and that left and right refer to the position of the projection, and not to the actual position of that part of the body, i.e., the left hand is the one that, once projected, is on the left side of the image. Through the custom dialog of the block, it is possible to skip the computation of some points; note that, however, since some points are computed relatively to other ones, there are some dependencies in the computations. For example, if the computation of the position of one shoulder is skipped, the position of the elbow and hand must be skipped too.

6.6 ManParamsCalcExtract Remarks

Extracts from the row vector given as input the values of the point selected through the ParamToExtract parameter (note that COG means Center Of Gravity, i.e. barycenter). The format of the input vector is the same as the vector generated by the FeatureCalc.centroidsCalc block.

6.7 Moment Parameters

Type of moment type of moment available in the IPL library

Remarks

Computes IPL spatial and central moments. The Type of moment parameter selects the type of computation, the mOrd and nOrd parameters specify its order. For further informtion please refer to par. 12-5 of the IPL 2.5 manual.

6.8 Moments Remarks

Computes all IPL spatial and central moments and outputs a 4x10 matrix containing the results. When computing more than one moment this block is probably more perfoming than separate FeatureCalc.Moment blocks. The matrix returned is as follows: the column indicates the type of moment (Spatial, Central, NormalizedSpatial, NormalizedCentral) and the row specifies the order of the moment: row0 : (m,n) = (0,0), row1 : (m,n) = {0,1), row2 : (m,n) = (0,2), row3 : (m,n) = (0,3),


row4 : (m,n) = (1,0), row5 : (m,n) = (1,1), row6 : (m,n) = (1,2), row7 : (m,n) = (2,0), row8 : (m,n) = (2,1), row9 : (m,n) = (3,0),

6.9 OpticalFlowLk Parameters

Load Load the image to be used as a reference

Remarks

Extracts the Horizontal and the Vertical components of the optical flow and returns them as output. Using the optical flow equation for a group of adjacent pixels and assuming that all of them have the same velocity, a system of linear equations is made. In a non-singular system, for two pixels, a velocity vector is computed to solve the system. However, for more than two pixels, combining equations is more effective, since it would be possible to get a system that has no solution, but that may yet be solved approximately, using the least square method with a weighted combination of equations. This method involves the solution of a 2 x linear system.

6.10 OpticalFlowPyrLk Parameters

No calc Enable | Disable the tracker

Remarks

Extracts the Horizontal and the Vertical components of the optical flow and returns them as output, tracking a point read in as the second input. Using the optical flow equation for a group of adjacent pixels and assuming that all of them have the same velocity, a system of linear equations is made. In a non-singular system, for two pixels, a velocity vector is computed to solve the system. However, for more than two pixels, combining equations is more effective, since it would be possible to get a system that has no solution, but that may yet be solved approximately, using the least square method with a weighted combination of equations. This method involves the solution of a 2 x linear system.

7 Draw

7.1 2PointElement Parameters

Type line | rectangle shape to be drawn

Remarks

Draws a line or a rectangle, depending on the Type parameter, on the output image. The image is read from the first input, while a 2x2 matrix is read from the second, containing either the start and end points of the line or the top left and bottom right corners of the rectangle. The Set color command allows to set the drawing color.

7.2 DrawPoint Parameters

Type cross | circle shape of marker


Remarks

Draws a point marker, depending on the Type parameter, on the output image. The image is read from the first input, while a 1x2 matrix is read from the second, containing the point coordinates. The Set color command allows to set the drawing color.

7.3 ManParams Parameters

DrawBRect true | false enable/disable drawing of bounding rectangle DrawCOGs true | false enable/disable drawing of cross markers Lines type of connecting mesh

Remarks

Draws the silhouette points computed by the FeatureCalc.ManParamsCalc and read as the second input. The image is read from the first input. The DrawBRect parameter, if checked, draws the bounding rectangle. The DrawCOGs parameter, when checked, draws cross markers on the computed points. The Lines parameter specifies the format of the connecting lines, None meaning that no lines are drawn. Central connects the barycenter to all the other point in a star fashion, while Hierarchical draws a skeleton-like mesh. The BRectColor, COGsColor and LinesColor commands allow to set the drawing color.

EyesWeb Sound Library Quick Reference 1

Sound Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the License provided.







Antonio Camurri DIST - University of Genoa Viale Causa, 13 16145 Genova, Italy E-mail: [email protected]


Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 1 Conversion.................................................................................................................................................. 3

1.1 BufferDescription ................................................................................................................................... 3 1.2 BufferSizeAdaptToClock ....................................................................................................................... 3 1.3 BufferSizeMultiplier............................................................................................................................... 3 1.4 ComposeChannels .................................................................................................................................. 3 1.5 ConvertFromMatrix ................................................................................................................................ 3 1.6 ConvertFromMatrixEx............................................................................................................................ 4 1.7 ConvertToMatrix .................................................................................................................................... 4 1.8 ExtractChannel ....................................................................................................................................... 4

2 Input............................................................................................................................................................ 4 2.1 Generator ................................................................................................................................................ 4 2.2 GeneratorExternalClock ......................................................................................................................... 5 2.3 WaveFileReader ..................................................................................................................................... 5 2.4 WaveFileReaderExternalClock .............................................................................................................. 5 2.5 WaveInput .............................................................................................................................................. 6 2.6 WaveInputExternalClock........................................................................................................................ 6

3 Output ......................................................................................................................................................... 6 3.1 WaveFileWriter ...................................................................................................................................... 6 3.2 WaveOutput............................................................................................................................................ 6

4 Operations................................................................................................................................................... 7 4.1 BinaryOp ................................................................................................................................................ 7 4.2 ChannelsMixer........................................................................................................................................ 7 4.3 FrameFeatures......................................................................................................................................... 7 4.4 SampleLevelDelay.................................................................................................................................. 8 4.5 SpectrumViewer ..................................................................................................................................... 8

5 Analysis ...................................................................................................................................................... 8 5.1 1DFFTp .................................................................................................................................................. 8 5.2 1DIFFTp ................................................................................................................................................. 8 5.3 FFT2Centroid ......................................................................................................................................... 9 5.4 FFT2En................................................................................................................................................... 9 5.5 IIRSoCell ................................................................................................................................................ 9 5.6 PitchEst_Corr.......................................................................................................................................... 9

6 VST plugins.............................................................................................................................................. 10 6.1 A sample module .................................................................................................................................. 10 Example ............................................................................................................................................................ 11

7 Datatypes .................................................................................................................................................. 11 7.1 1DFftBuffer .......................................................................................................................................... 12 7.2 GetFftBuffer ......................................................................................................................................... 13 7.3 GetNumChannels, GetSamplingFrequency, GetFftSize, GetFftOrder, GetAudioBufferSize .............. 14 7.4 SoundBuffer.......................................................................................................................................... 14 7.5 ConvertFromPCM ................................................................................................................................ 15 7.6 ConvertToPCM..................................................................................................................................... 15 7.7 CopyIntervalFrom................................................................................................................................. 16 7.8 GetBuffer .............................................................................................................................................. 16 7.9 GetNumChannels, GetNumSamples, GetSamplingFrequency............................................................. 16 7.10 GetSample ............................................................................................................................................ 17 7.11 SetSample ............................................................................................................................................. 17

8 References ................................................................................................................................................ 17


Sound Library The Sound Library is contained in the EywSound.dll, EywSoundDatatype.dll and EywSoundExtras.dll DLL files. It contains blocks for sound processing.

1 Conversion

1.1 BufferDescription Remarks

Extracts information about the input buffer. NumChannels contains the number of channels in the buffer; NumSamples the number of samples; SamplingFrequency its sampling frequency. Note that these are all design-time properties, thus, the output will remain constant during the whole execution; the output value will nevertheless be refreshed for each new input buffer.

1.2 BufferSizeAdaptToClock Remarks

Reduces the buffer size of the input audio stream, in order to adapt it to the input clock. The first input is a clock signal, the second input is the audio stream. The output audio stream has a buffer size coherent with the input clock. Note that only decreasing the buffer size by an integer factor is currently supported.

1.3 BufferSizeMultiplier Remarks

Increases the buffer size by an integer factor. When working in Overlapping mode an output buffer is generated for every input buffer, disregarding the Factor parameter: the resulting output clock is not coherent with its buffer size and sampling frequency. Hence, the resulting audio stream must not be used to obtain a clock signal.

1.4 ComposeChannels Remarks

Creates an output sound buffer with a number of channels equal to the sum of the two input buffers. If the first input has N channels, and the second has M, then the output will have N + M channels. Output channels 0, 1, ..., N-1 come from the first input, channels N, N+1,..., N + M - 1 from the second.

1.5 ConvertFromMatrix Parameters

Frequency output frequency

Remarks

Converts the input matrix to the sound buffer datatype. Each column of the input matrix is interpreted as a sound channel. Note that the frequency of the output buffer must be specified with the Frequency parameter.


1.6 ConvertFromMatrixEx Remarks

Converts the input matrix to the sound buffer datatype. Each column of the input matrix is interpreted as a sound channel.The format (number of channels, number of samples, sampling frequency) of the output datatype is read from the second input, which is used only for this purpose during the Init phase, while it is ignored during execution.

1.7 ConvertToMatrix Remarks

Converts the input sound buffer to the matrix datatype. Each column of the output matrix corresponds to a channel of the input sound buffer.

1.8 ExtractChannel Remarks

Extracts the requested channel from the input sound buffer. The Channel parameter specifies which channel to copy into the output buffer.

2 Input

2.1 Generator Parameters

Sampling frequency sampling frequency of the generated audio stream Custom sampling frequency

manually editable value, if a non-standard sampling frequency is desired

Buffer size size, in samples, of each generated buffer Custom buffer size manually editable value, if a non-standard buffer size is

desired WaveForm type type of waveform Unused name and interpretetion of these parameters depends on the

value of the waveform type combo box Unused name and interpretetion of these parameters depends on the

value of the waveform type combo box Unused name and interpretetion of these parameters depends on the

value of the waveform type combo box Remarks

Generates a waveform. The type of waveform is selected by the WaveForm Type parameter. Each type of generated sound needs a variable number of parameters. Hence, the interpretation of the parameters following the waveform type combo-box, depends on the value of the combo itself. See the remarks section for more details. The Generator block manages different types of waveforms:

• Silence generates a silent audio stream, that is, all generated samples are zero. • Sinusoidal generates a sinusoidal waveform. The parameters let you specify:

o Frequency (Hz) frequency, in hertz, of the generated sinusoidal signal. o Phase (Degrees) phase, in degrees, of the sinusoid.

• Square wave generates a squared waveform. The generated samples have value 0 (down) or 1 (up). o Frequency (Hz) frequency, in hertz, of the generated sinusoidal signal.


o Phase (Degrees) phase, in degrees, of the sinusoid. o Perc. up percentage of the up phase compared to the down phase in each period. A

value of 0.0 means that are zero (down level); a value of 1.0 means that all samples are 1.0 (up level); intermediate values cause the signal to switch between the two levels.

• Constant values generates a waveform where all samples have the same constant value: o Value value to assign to the output samples.

• Rand Uniform generates a waveform where each sample have a random value. The value is chosen from a uniform distribution, where the minimum and maximum value are:

o Min minimum value of the uniform distribution o Max maximum value of the uniform distribution

• Square wave (zero crossing) generates a squared waveform. The generated samples have value –1.0 (down) or 1.0 (up). The difference with the Square wave option is that this waveform crosses the zero level.

o Frequency (Hz) frequency, in hertz, of the generated sinusoidal signal. o Phase (Degrees) phase, in degrees, of the sinusoid. o Perc. up percentage of the up phase compared to the down phase in each period. A

value of 0.0 means that are zero (down level); a value of 1.0 means that all samples are 1.0 (up level); intermediate values cause the signal to switch between the two levels.

2.2 GeneratorExternalClock Remarks

Same as the Generator block, except for an additional input of type PeriodicClock (Clock Library), used for external synchronization. The Buffer size parameter here is not necessary since its value is derived from the clock information.

2.3 WaveFileReader Parameters

Filename WAV file name and path Buffer size output buffer size FileBufferSize file cache size Continuous enable/disable continuous output Loop enable/disable autorestart RewindOnFileChange Determines the behaviour when changing file

Remarks

Reads a standard Windows wavefile. The Filename parameter indicates the name of the file to be read; ActivationMode selects the type of activation: periodic for normal usage or OnEvent to cause the next buffer to be read only when a ReadFile command is received. The BufferSize parameter specifies the size of the output buffer, in milliseconds, while FileBufferSize specifies the size of the internal file cache. The Continuos parameter, when checked, sends out data even when the block is paused, while Loop, when checked, automatically restarts playing from the beginning when EOF is reached. It is possible to change the name of the file even during the execution of the patch. In this case the new file can either start from the beginning on from the current file position of the old one. The behavior can be controlled by means of the parameter RewindOnFileChange (true: start from the beginning; false: keep current file position). Note that the new file must be compatible with the old one, i.e., it must have the same sampling frequency and number of channels.

2.4 WaveFileReaderExternalClock Remarks

Same as the WaveFileReader block, except for an additional input of type PeriodicClock (Clock Library), used for external synchronization. The Buffer size parameter here is not necessary since its value is derived from the clock information.


2.5 WaveInput Parameters

Device device ID Buffer size output buffer size Sampling frequency input sampling frequency Bits per sample sample resolution Num channels number of channels

Remarks

Captures waveform audio from the specified device using standard Windows Multimedia APIs. The Device parameter allows to select the input device to use, among the installed ones. BufferSize specifies the size of the output buffer, in milliseconds; Num buffers specifies the number of internal buffers to use: two are enough in most situations; increasing this number can reduce clicks, without causing delays. This leads to an higher memory usage, though, and may degrade the performance of the system. Sampling Frequency specifies the sampling frequency at which to capture sound: any value in the range [100Hz, 100.000Hz] may be specified, though, some devices might support only the standard frequencies (44.100, 22.050, 11.025). Bits Per Sample specifies the number of bits per sample to be used: note that the output will always be converted to a double in the range [-1.0, +1.0]. Num Channels specifies the number of channels to use.

2.6 WaveInputExternalClock Remarks

Same as the WaveInput block, except for an additional input of type PeriodicClock (Clock Library), used for external synchronization. The Buffer size parameter here is not necessary since its value is derived from the clock information.

3 Output

3.1 WaveFileWriter Parameters

Filename WAV file name and path Bits per sample sample resolution Num of channels number of output channels Left channel index left channel ID Right channel index right channel ID

Remarks

Saves the input audio stream on a standard wavefile. The Filename parameter indicates the name of the file to write; Bits per sample specifies the resolution of each saved sample; Num of channels specifies the number of channels (1 for mono, 2 for stereo); Left channel index specifies which of the input channels has to be saved as the left channel (or the unique mono channel, according to the value of Num of channels), while Right channel index does the same for the right channel and is.disabled for monophonic audio.

3.2 WaveOutput


Parameters

Device device ID Bits per sample sample resolution Num of channels number of output channels Left channel index left channel ID Right channel index right channel ID

Remarks

Plays waveform audio on the specified device using standard Windows Multimedia APIs. The Device parameter allows to select the input device to use, among the installed ones.Bits per sample specifies the resolution of the data sent to the audio device; Num of channels specifies the number of channels (1 for mono, 2 for stereo); Left channel index specifies which of the input channels has to be output as the left channel (or the unique mono channel, according to the value of Num of channels), while Right channel index does the same for the right channel and is disabled for monophonic audio.

4 Operations

4.1 BinaryOp Parameters

Operation Type type of operation

Remarks

Performs a binary operation on the two input buffers. The buffers must be of the same type, i.e. they must have the same length, number of channels, and sampling frequency. The type of the operation is selected with the Operation Type parameter, No operation meaning that the first input is output directly without modification. Add, Sub and Multiply operate between the corresponding audio samples; SubInv works like Sub but inverting the order of the operands (i.e. B - A instead of A - B).

4.2 ChannelsMixer CLSID: 0x1a22967b,0x7e55,0x4016,{0x82,0xf7,0x66,0x54,0xae,0x49,0x7f,0x65}

Inputs

Input sound stream IDTSoundBuffer <Notes> Sliders IDTMatrix <Notes>

Outputs

Output sound stream IDTSoundBuffer <Notes>

Remarks

This blocks provides a multichannel mixing mechanism. The input can be a sound stream with any number of channels (i.e., mono, stereo, multichannel, etc.); the output stream has a number of channels equal to the number of rows of the input matrix (Sliders). Each output channel is the weighted sum of the input, where the weights are the positions of the sliders. Thus the input matrix must have a number of columns equal to the number of channels of the input sound stream. The output audio streams are normalized by assuming that each value of the input matrix is in the range [0.0, 1.0].

4.3 FrameFeatures Parameters

OpType type of operation


Channel audio channel

Remarks

Computes frame-level features on the input sound stream and outputs the result as a scalar real number. The OpType parameter specifies the feature to be computed: Volume is computed as v = sqrt ( sum ( ( s(i) * s(i) ) / N ), where N is the number of samples of the buffer, and i is in the range [0...N-1]; ZeroCrossing contains the number of zero crossings in the input buffer. The Channel parameter specifies the channel for which the specified feature has to be computed

4.4 SampleLevelDelay Parameters

Measurement Unit delay unit

Remarks

Delays the input sound stream of the specified number of samples (or milliseconds, according to the Measurement Unit parameter). Note that the delay is implemented by shifting the input stream of the necessary number of samples, and not delaying it in time. The initial samples are set to zero.

4.5 SpectrumViewer Parameters

Measurement Unit delay unit

Remarks

Draws the spectrum of the 1DfftBuffer input. The spectrum is drawn in the custom dialog of the block.

5 Analysis

5.1 1DFFTp Parameters

FFT size length of the FFT buffer

Remarks

Performs the computation of the one-dimensional FFT transform of the input audio stream. The size of the input buffer must be less than the order of the FFT. That is, supposing that the FFT size is 4096 (the maximum allowed value), the size of the input buffer must be at most 4096 samples. It is up to the user to build a patch such that this constraint is respected. The output of this block is a 1DfftBuffer datatype containing the real and imaginary components of the result of the transformation.

5.2 1DIFFTp Remarks

Given a 1DfftBuffer as input, computes the inverse FFT transformation, in order to obtain the audio buffer representation in the time domain.


5.3 FFT2Centroid Parameters

Channel select the channel on which to compute the centroid Threshold energy threshold in decibel

Remarks

Computes the centroid of the input spectral signal. The centroid is computed on a single channel, that may be choosen by means of the channel parameter.

5.4 FFT2En Remarks

Given a 1DfftBuffer as input, computes the energy of the buffer. The output is a scalar number. The input must be a single-channel FFT (use the ExtractChannel block on the audio stream).

5.5 IIRSoCell Parameters

Fc band center Bc bandwidth

Remarks

Second order IIR filter. Acts as a band-pass filter on the input audio stream, and works in the time domain. The output is the filtered signal. The parameters fc and bc represent the centre of the band, and the width. Example

The following patch uses a band-pass filter (IIRSOCell) to modify the heard audio signal, and an FFT transformation followed by a SpectrumViewer block to visualize the spectrum of the original signal.

5.6 PitchEst_Corr


Parameters

channel select the channel on which to compute the pitch filter …

Remarks

Computes an estimate of the pitch of the input audio signal, and outputs it as a scalar value. The estimate is computed using the autocorrelation of the input signal. The block works on a single channel, thus, the channel on which to operate must be selected through the channel parameter, in the case of multichannel sound signals.

6 VST plugins VST (Virtual Studio Technology) Plugins are add-in modules originally designed to be hosted inside Steinberg’s audio programs like Cubase. By creating such ad-hoc modules, it was possible to extend the capabilities of the application beyond Cubase programmers’ imagination; and, given the number of third-party plugins now available on the market (both commercial and free), the audio composer can quickly find the effect he needs. The EywVST module allows EyesWeb users to take such plugins and make them part of a patch. The EywVST module is a generic host, that needs to be made aware of the VST plugins it must expose to EyesWeb; so it expects them to be placed in a VSTPlugins directory located in the same directory where the binary EywVST.dll will be placed. Moreover, it also searche for plugins located in the place where Steinberg’s audio programs search them (usually: C:\Program Files\Steinberg). Each time the EywVST.dll file is registered, a certain number of EyesWeb modules are generated, corresponding to the number of VST plugins found in the system. This means that if new plugins are added to the system, the dll must be registered again to make EyesWeb aware of the new plugins. For registering the EywVST.dll file, simply execute the batch file RefreshVSTplugins.bat that is located in the same directory where the EyesWeb executable file is (the default location is C:\Program Files\EyesWeb 2.4\Binaries). Note that EyesWeb must be quit and restarted again to see the changes in the number of available VST plugins (or ReloadLibraries must be chosen from the System menu, when no patch is opened). Anyway, note that these operations are automatically performed during the installation of EyesWeb, hence, once EyesWeb is installed, you should see the plugins that where already installed on your system. At least, you should see the NorthPole plugin, that is distributed together with EyesWeb, and that is available, free of charge, from Prosoniq at http://www.prosoniq.net/pub/freebies/pc/NPOLE.ZIP.

6.1 A sample module The EywVST module usually has an input and an output connection of type SoundBuffer, even if, depending on the characteristics of the hosted VST plugin, it could have just an input (e.g. an output VST plugin) or just an output (e.g. a sound generator VST plugin) connection. The input connection could be either mono or stereo. The parameter page for the module shows the parameter that the VST plugin exposes: the value they can assume is always a float between 0.0 and +1.0. The following is a page sample:

If the VST plugin exposes a custom GUI interface to better edit these parameters, it is shown in the “Custom” property page of the EywVST module. The following is exposed by the NorthPole VST plugin:


Example The simplest patch is one where a .WAV file is read from the file system, processed by the VST plugin (manipulating its parameters from the custom property page in real time, if necessary) and played.

7 Datatypes This part of the guide describes the programming interface of the Sound library, and can therefore be skipped by users who do not plan to develop their own blocks.

The Sound library comes with two different datatypes, one to handle signal in the time domain, the other to handle signals in the frequency domain.

To use the MIDI datatypes you must include EyesWebSound.h or EyesWebSoundAnalysis.h from within your files (you can include them in your stdafx.h, to avoid having to include them in all files). Moreover, the definition of the CLSIDs and IIDs of the Sound datatypes could turn out to be useful. The available CLSIDs are:

const IID IID_IDT1DFftBuffer = {0x9E143AF0,0x08E0,0x4347,{0xA3,0xA5,0x43,0xE3,0xC8,0xA3,0xCC,0x84}};


const CLSID_DT1DFftBuffer = {0x50A6009C,0x2A70,0x4efe,{0x80,0x3F,0x4E,0x26,0xE7,0x85,0x79,0x07}};

const IID IID_IDTSoundBuffer = {0x55B9A12A,0xF5E2,0x421D,{0x98,0xE2,0xC5,0xC8,0x81,0xFB,0x3E,0xF3}};

const CLSID CLSID_DTSoundBuffer = {0x1937EAF5,0x5BD8,0x4FF9,{0x88,0x50,0x9F,0x52,0x06,0x25,0x74,0x11}};

These declarations can be placed somewhere in your cpp files, e.g., in the FactoryData.cpp file. They can be used instead of the string identifiers placed by the Wizard in the template code of your blocks. For instance, when a new block is generated by the Wizard using the SoundBuffer datatype, some piece of code like the following is generated:

SetInputInfo( INPUT_INDEX, "input_name", "{1937EAF5-5BD8-4FF9-8850-9F5206257411}", "{55B9A12A-F5E2-421D-98E2-C5C881FB3EF3}", EYWIF_REQUIRED_INITIALIZATION | EYWIF_REQUIRED_EXECUTION );

where the two string identifiers are, respectively, the CLSID and the IID of the SoundBuffer datatype. The two identifiers could be replaced manually by the developer with the previously declared variable, in order to obtain the more readable code:

SetInputInfo( INPUT_INDEX, "input_name", CLSID_DTSoundBuffer, IID_IDTSoundBuffer, EYWIF_REQUIRED_INITIALIZATION | EYWIF_REQUIRED_EXECUTION );

Anyway, the proposed operation is just for the sake of readability of the code, and is not necessary for the proper working of the block. Moreover, the same operation can be done in an analogous manner for the outputs of the block (the example refers to an input), and for the 1DFftBuffer datatype also. More generally, the same operations could be performed whenever a CLSID or IID of a datatype is used for developing EyesWeb blocks.

The datatypes that are described in the following sections provide a custom COM interface that inherits from the IEyesWebDatatype interface, which on its own, inherits from IUnknown. Only the custom interfaces are described in this guide, as the IEyesWebDatatype interface is described in the EyesWeb Programmers Guide, whereas the IUnkown interface is described in the Microsoft MSDN™ library (see [1]), in every COM-related book, and in several other sources.

7.1 1DFftBuffer

The 1DFftBuffer datatype implementation is contained in the EywSoundAnalysisDatatype.dll file, and EyesWebSoundAnalysis.h must be included in order to use it. This datatype contains a buffer of the representation of the audio signal in the frequency domain, that is, decomposed in its real and imaginary parts. Hence, each sample of data is a structure containing two fields:

typedef struct _SCplx { float re; float im; } SCplx;

Clearly, the first field represents the real part, whereas the second field contains the imaginary part. The above definition is taken from the file nsp.h, that is needed to use the 1DfftBuffer. The nsp.h file is part of the Intel® Signal Processing Library (shortly, SPL), that can be freely downloaded from the Intel website (http://www.intel.com). Note that the SPL is distribuited together with EyesWeb, since version 2.3.1. Anyway, the distributed files only include the run-time components; the development file (such as the nsp.h header) are only included in the complete package.

Multi-channel audio sampled data. The length of the buffer (in number of samples), the sampling frequency and the number of channel must be specified in the initialisation phase, and are assumed constants throughout a run. Each sample of the audio signal is represented by means of a float number. Although the samples can assume any value, you should be aware that the input and output blocks of the Sound library assume that the data is in


the range [-1.0, 1.0] when converting from/to the PCM format (the most common format supported by audio devices).

Interface and structures

typedef struct { int iReserved; DWORD dwNumChannels; DWORD dwSamplingFrequency; DWORD dwHopSize; DWORD dwFftSize; DWORD dwFftOrder; DWORD dwAudioBufferSize; } EYW_1DFFTBUFFER_INIT_DATA;

interface IDT1DFftBuffer : IeyesWebDatatype { HRESULT GetFftBuffer( DWORD dwNumChannel, SCplx **ppdFftBuffer ); HRESULT GetNumChannels( DWORD *pdwNumChannels ); HRESULT GetSamplingFrequency(DWORD* pdwSamplingFrequency); HRESULT GetFftSize(DWORD* pdwFftSize); HRESULT GetFftOrder(DWORD* pdwFftOrder); HRESULT GetAudioBufferSize(DWORD* pdwAudioBufferSize); };

CLSID : 0x50A6009C,0x2A70,0x4efe,{0x80,0x3F,0x4E,0x26,0xE7,0x85,0x79,0x07}; IID : 0x9E143AF0,0x08E0,0x4347,{0xA3,0xA5,0x43,0xE3,0xC8,0xA3,0xCC,0x84};

Remarks

The 1DFftBuffer datatype needs to be initialised by means of the EYW_1DFFTBUFFER_INIT_DATA structure. The first field of the structure must be set to zero. The other fields represent, respectively, the number of channels, the sampling frequency of the sound buffer from which this FFT is derived, the hop size of the original sound buffer (when equal to dwAudioBufferSize it means that buffers do not overlap), the Fft size and order, and the buffer size of the original audio signal (dwAudioBufferSize),

Example 1 – a block generating an FFT buffer, starting from a stereo, 44100Hz, audio signal.

EYW_1DFFTBUFFER_INITDATA initData; initData.iReserved = 0; initData.dwNumChannels = 2; initData.dwSamplingFrequency; // 44100 initData.dwHopSize = 4000; // no overlapping initData.dwFftSize = 4096; initData.dwFftOrder = 12; // 4096 = 2^12 initData.dwAudioBufferSize = 4000; // shorter than FFT Í 0 padded SetOutputInitData( 0, &initData );

where the above code refers to the Init method of the block.

7.2 GetFftBuffer Parameters DWORD dwNumChannel number of channels SCplx **ppdFFtBuffer pointer to a variable that will receive a pointer to the

buffer Remarks The GetFftBuffer method allows to get a pointer to the array of SCplx samples for the specified channel. It is noteworthy that you just have to supply the address of a variable of type SCplx *, and not the whole buffer. The


datatype returns a pointer to its internal buffer, hence, the user of the datatype does not need to allocate/deallocate memory. For instance, you can use the GetFftBuffer method as follows: Example 1 – reading the content of the FFT buffer.

HRESULT CmyBlock::Execute( DWORD, DWORD ) { SCplx *pFftBuffer; IDT1DfftBuffer *pBuffer = (IDT1DfftBuffer *)(LockInput(0));

7.3 GetNumChannels, GetSamplingFrequency, GetFftSize, GetFftOrder, GetAudioBufferSize

Parameters DWORD *pdwXXXX pointer to a DWORD variable that will receive the

requested value Remarks These methods simply return, in the DWORD whose pointer is supplied as argument, the value of the field of the EYW_1DFFTBUFFER_INIT_DATA structure. PBuffer->GetFftBuffer( &pFftBuffer ); for( DWORD s = 0; s < dwFftSize; ++s ) { pFftBuffer[s].re = 0.0f; pFftBuffer[s].im 0.0f; } UnlockInput(0); return S_OK; }


7.4 SoundBuffer

The SoundBuffer datatype implementation is contained in the EywSoundDatatype.dll file, and EyesWebSound.h must be included in order to use it. This datatype contains a buffer of multi-channel audio sampled data. The length of the buffer (in number of samples), the sampling frequency and the number of channel must be specified in the initialisation phase, and are assumed constant throughout a run. Each sample of the audio signal is represented by means of a float number. Although the samples can assume any value, you should be aware that the input and output blocks of the Sound library assume that the data is in the range [-1.0, 1.0] when converting from/to the PCM format (the most common format supported by audio devices).


typedef struct { DWORD dwNumChannels; DWORD dwMaxNumSamples; DWORD dwSamplingFrequency; } EYW_SOUNDBUFFER_INIT_DATA

interface IDTSoundBuffer : IeyesWebDatatype { HRESULT GetBuffer( DWORD dwNumChannel, float **ppdBuffer ); HRESULT GetNumChannels( DWORD *pdwNumChannels );


HRESULT GetMaxNumSamples( DWORD *pdwMaxNumSamples ); HRESULT GetSamplingFrequency( DWORD *pdwSamplingFrequency ); HRESULT SetSample( DWORD dwChannel, DWORD dwSampleIdx, float dSample ); HRESULT GetSample( DWORD dwChannel, DWORD dwSampleIdx, float *pdSample ); HRESULT ConvertToPCM( DWORD dwSourceChannel, DWORD dwDestPCMChannel, void *pWaveFormatEx, void *pBuffer ); HRESULT ConvertFromPCM( DWORD dwSourcePCMChannel, DWORD dwDestChannel, void *pWaveFormatEx, void *pBuffer ); HRESULT SetInterval( DWORD dwFirstSampleIdx, DWORD dwEndSampleIdx, float dVal ); HRESULT CopyIntervalFrom( DWORD dwDestFirstSampleIdx, DWORD dwSourceFirstSampleIdx, DWORD dwSourceEndSampleIdx, IDTSoundBuffer *pDTSourceBuffer ); };

CLSID : 0x55B9A12A,0xF5E2,0x421D,{0x98,0xE2,0xC5,0xC8,0x81,0xFB,0x3E,0xF3}; IID : 0x1937EAF5,0x5BD8,0x4FF9,{0x88,0x50,0x9F,0x52,0x06,0x25,0x74,0x11};

Remarks

The SoundBuffer datatype needs to be initialized by means of the EYW_SOUNDBUFFER_INIT_DATA structure. The fields of the structure specify, respectively, the number of channels, the number of samples per buffer, and the sampling frequency.

For example, a block generating a standard stereo audio signal, sampled at 44100Hz, and with a buffer size equal to 1s (that is, 44100 samples), will initialise its output data as in the following example.

Example 1 – a block generating a stereo, 44100Hz, audio signal

EYW_SOUNDBUFFER_INITDATA initData; initData.dwNumChannels = 2; initData.dwNumSamples = 44100; // that is, 1 second initData.dwSamplingFrequancy = 44100; SetOutputInitData( 0, &initData );


7.5 ConvertFromPCM Parameters DWORD dwSourcePCMChannel zero-based channel index in the PCM buffer DWORD dwDestChannel zero-based index of the destination samples void *pWaveFormatEx pointer to a structure describing the format of the PCM

data void *pBuffer pointer to the PCM buffer Remarks

This method provide the conversion from the PCM format to the float type, used internally by the SoundBuffer datatype. The conversion from both the 16 bit format or the 8 bit format is available. pBuffer is the pointer to the PCM buffer that contains the data to be converted; dwSourcePCMChannel specifies the index of the channel to be read from the source PCM buffer.

7.6 ConvertToPCM Parameters DWORD dwSourceChannel zero-based channel index


DWORD dwDestPCMChannel zero-based index of the destination PCM samples void *pWaveFormatEx pointer to a structure describing the format of the PCM

data void *pBuffer pointer to the PCM buffer Remarks This method provides the conversion from the float type, used internally by the SoundBuffer datatype, to the PCM format. The conversion to both the 16 bit format or the 8 bit format is available. pBuffer is the pointer to the PCM buffer that will receive the converted samples. Note that only the specified channel will be written, thus, if, for instance, the destination buffer has two channels, only the samples belonging to one of the two will be written, the other samples will be left unchanged.

7.7 CopyIntervalFrom Parameters DWORD dwDestFirstSampleIdx zero-based index of the first destination sample DWORD dwSourceFirstSampleIdx zero-based index of the first source sample DWORD dwSourceEndSampleIdx zero-based index of the last source sample (unread) IDTSoundBuffer *pDTSourceBuffer pointer to the source SoundBuffer Remarks

This method allows one to easily copy a subset of the samples of a SoundBuffer, to another SoundBuffer instance. The length of the buffer copied is specified by means of the dwSourceFirstSampleIdx and dwSourceEndSampleIdx; keep into account the fact that the dwSourceEndSampleIdx sample is used as a delimiter, but not as a source sample, i.e., that sample is not copied. dwDestFirstSampleIdx specifies the starting destination position

7.8 GetBuffer Parameters DWORD dwNumChannel number of channels float **ppdBuffer pointer to a variable that will receive a pointer to the

buffer Remarks The GetBuffer method allows to get a pointer to the array of float samples for the specified channel. It is noteworthy that you just have to supply the address of a variable of type float *, and not the whole buffer. The datatype returns a pointer to its internal buffer, hence, the user of the datatype does not need to allocate/deallocate memory.

7.9 GetNumChannels, GetNumSamples, GetSamplingFrequency Parameters DWORD *dwXXXX pointer to a variable that will receive the return value Remarks


These methods return, respectively, the number of channels, the number of samples, and the sampling frequency. The values returned are exactly the same used when initialising the datatype.

7.10 GetSample Parameters DWORD dwChannel zero-based channel index DWORD dwSampleIdx zero-based sample index float *pfVal pointer to a float variable, that will receive the value of the

sample Remarks This method retrieves the value of a specific sample. Both dwChannel and dwSampleIdx are zero-based indexes, and they must be in the range [0, GetNumChannels()-1] and [0, GetNumSamples()-1] respectively. This method might be useful when reading a small subset of the samples of the buffer; when a large amount of data has to be read, consider the possibility to use the GetBuffer method to get a pointer to the data, and then directly accessing the array. In facts, although simpler, the GetSample method could badly affect the performance due to the overhead of a function call per sample.

7.11 SetSample Parameters DWORD dwChannel zero-based channel index DWORD dwSampleIdx zero-based sample index float fVal new value of the sample Remarks This method let you set the value of a specific sample. Both dwChannel and dwSampleIdx are zero-based indexes, and they must be in the range [0, GetNumChannels()-1] and [0, GetNumSamples()-1] respectively. This method might be useful when setting a small subset of the samples of the buffer; when a large amount of data has to be written, consider the possibility to use the GetBuffer method to get a pointer to the data, and then directly accessing the array. In facts, although simpler, the SetSample method could badly affect the performance due to the overhead of a function call per sample.

8 References

[1] Microsoft MSDN™ Library: http://msdn.microsoft.com

EyesWeb MIDI Library Quick Reference 1

MIDI Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the Licence provided.









Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 1 Conversions ................................................................................................................................................ 4

1.1 BufferToMessage ................................................................................................................................... 4 1.2 MessageToBuffer ................................................................................................................................... 4 1.3 MessageToScalar.................................................................................................................................... 5 1.4 ScalarToMessage.................................................................................................................................... 5

2 Input............................................................................................................................................................ 5 2.1 BufferFileReader .................................................................................................................................... 5 2.2 BufferInput ............................................................................................................................................. 6 2.3 Input........................................................................................................................................................ 6 2.4 MessageInput.......................................................................................................................................... 7

3 Output ......................................................................................................................................................... 7 3.1 MessageOutput ....................................................................................................................................... 7 3.2 Output ..................................................................................................................................................... 7

4 Operations................................................................................................................................................... 8 4.1 BufferDescription ................................................................................................................................... 8 4.2 MessageCheckDuplicates ....................................................................................................................... 9 4.3 MessageField .......................................................................................................................................... 9 4.4 MessageFilter........................................................................................................................................ 10 4.5 MessageGenerator - MessageGeneratorSyncIn .................................................................................... 10 4.6 NoteOn2NoteOff .................................................................................................................................. 11

5 Datatypes .................................................................................................................................................. 11 5.1 MIDIBuffer........................................................................................................................................... 12 5.2 MIDIMessage ....................................................................................................................................... 13

6 References ................................................................................................................................................ 14


MIDI Library The MIDI Library is contained in the EywMidi.dll and EywMidiDatatype.dll DLL files. It contains blocks for MIDI (Musical Instrument Digital Interface, see [1]) processing. The library includes blocks for input and output of MIDI data through the available devices; blocks to handle and modify data are also provided, as are blocks that generate MIDI sequences. Furthermore, MIDI files can be used both as a source or a destination of the EyesWeb-managed MIDI data.

MIDI data are handled in three different ways inside the MIDI library:

(a) MIDI data may be treated like simple scalar numbers. This is the approach used by the MIDI.Input and MIDI.Output blocks; these blocks deal with integer scalar numbers where only the least significant byte is used, and they interpret that byte as the Byte2 of MIDI messages. The other MIDI fields (i.e., the Status byte and the Byte1) are specified by means of the parameters of the blocks;

(b) MIDI data may be treated like a structure containing all fields, i.e. the Status byte, Byte1, and Byte2. This is the way most of the blocks in the MIDI library work; usually the names of these blocks have the word “Message” as a part of the name.

(c) MIDI messages may finally be handled not as single commands, but as buffers (or sequences) of commands. This approach is especially useful when working together with audio, or with any other data that inside EyesWeb is managed in a buffered fashion. Hence, the handling of MIDI as groups of commands allows to easily associate a set of MIDI messages to the corresponding buffer of audio data (usually the correspondence is intended as a temporal interval coincidence).

In the remaining part of the documentation the terms “MIDI message” and “MIDI commands” will be used interchangeably, and they will both refer to a 32bit message, where only the first three bytes are used, and are named as follow:

first byte Status byte

second byte Byte1

third byte Byte2

fourth byte (ignored)

A MIDI input/output device is required to use all input/output blocks. Currently (version 2.3), the system may crash if no MIDI device is found. The output blocks of the MIDI library display a combo box where it is possible to choose which device to use; the first device should usually be the MIDI mapper (the exact name may vary from system to system): this is a virtual device provided by Windows, that can be remapped to one of the other devices through the Windows Control Panel.

Compatibility warning: prior to the 2.3.1 version of the MIDI library, MIDI channels where numbered from 0 to 15. From this version on, the channel number may assume values from 1 to 16. When you open previously created patches, the stored value of the channel number will be in the old range, hence, may be you have to adjust it by adding 1 to its value. Moreover, if the patch was stored with 0 as channel number, EyesWeb will recognize that the parameter is wrong when the patch is loaded, and will raise a warning message.


1 Conversions

1.1 BufferToMessage CLSID: 0x7a1ffa3c,0x0185,0x4a30,{0x81,0x2e,0x01,0xf0,0x27,0x54,0xe3,0xc2}

Inputs

MIDICommandsBuffer MIDI.MIDIBuffer

Outputs

MIDIMessage MIDI.MIDIMessage

Parameters

Delay Integer

Remarks

Receives a buffer of MIDI messages as input, and outputs the single messages with the correct timings. The timestamps associated to each message are used for recreating the correct sequence of messages. The Delay parameter allows to force a constant delay to the whole stream: when the input buffer is received, the messages are not sent out immediately according to their timestamp, but the specified delay is added to all messages. Hence, even if the successive input buffer arrival time is not precise the output stream has no gaps, provided that the delay of the input is less then the delay specified by this parameter. This may help to solve timing issues, especially when the system is overloaded or when the OS is less precise (as it is in the case of Win98, if compared to the Win2k timing mechanism). Note for programmers All timestamps are relative to the timestamp of the first MIDI message (in the first buffer), i.e., if the first MIDI message has timestamp equal to 0, all successive timestamps are interpreted in a straightforward manner, otherwise all successive timestamps are obtained subtracting the value of the first timestamp. Beside this exception, the other messages need not to appear ordered by timestamps. Anyway, note that the timestamp 0 is interpreted as the beginning of a new stream.

1.2 MessageToBuffer

CLSID : 0xbcf73090,0xaf8e,0x4e23,{0xaf,0xf8,0x61,0xdf,0x3e,0x2f,0xa3,0x08}

Inputs

SyncIn Any MIDIMSG Midi.MidiMessage

Outputs

MIDIBuffer Midi.MidiCommandsBuffer

Parameters

Max buffer size Integer [0,+ ∞)

Remarks

Groups the incoming MIDI messages in a buffer. Each message is copied to the buffer with its own timestamp. The first input of the block is used as a synchronization signal, and it determines the starting of a new buffer. Thus, the buffer is synchronized with the clock of that data stream. Note that the accepted datatype on that input is Any, thus, every link is a valid clock signal for this block. The second intput is the stream of MIDI messages that are collected into MIDI buffers; the output is the created buffer. The parameter Max buffer size establishes the maximum length (expressed in number of MIDI messages) of the buffer. When the buffer sizes exceeds this value, all successive messages are ignored until the buffer is reset to a


zero length (and this happens when a new clock signal is generated on the first input). A value of zero for this parameter means that no upper bound exists.

1.3 MessageToScalar

CLSID : 0x5b065faa,0xea0d,0x4745,{0xa9,0x0d,0x57,0xbd,0x68,0x09,0x67,0x11}

Inputs

MIDIMSG Midi.MidiMessage

Outputs

Scalar MathDatatype.Scalar

Remarks

Converts the MIDI message to a scalar integer number (32 bits). The integer number can be used in any mathematical operation performed by the Scalar blocks of the Math library. The output scalar integer number is made up of four bytes, where, from the least significant to the most significant, the first byte is the Status byte, the second byte is Byte1 of the MIDI message, the third byte is Byte2, the fourth byte is zero.

1.4 ScalarToMessage

CLSID : 0x553aa90a,0xbe13,0x453a,{0xa9,0xf1,0x62,0x09,0xb2,0x8f,0xb9,0xd7}

Inputs

Scalar MathDatatype.Scalar

Outputs


Remarks

Converts the input scalar integer number to a MIDI message. The most significant byte is ignored; the other bytes, starting from the most significant, are interpreted as Byte2, Byte1, and the Status byte.

2 Input

2.1 BufferFileReader

CLSID: 0xc73ee6a4,0x3556,0x4a1d,{0x96,0xae,0x89,0xbb,0x86,0xa0,0x79,0xfa}

Inputs

Clock Clock.PeriodicClock

Outputs

MIDI Commands buffer Midi.MidiBuffer End of file Bang

Parameters

Filename Name of the MIDI (*.mid) file


Remarks

This block reads a standard MIDI file (*.mid), and outputs it in buffers of MIDI messages. This block is designed to be synchronized with an external clock source. This allows, for instance, to synchronize the reading of a MIDI file to a sound stream. Note that the input is a periodic clock, and the period is assumed to be constant: supposing that the period is 40 ms, at the first activation the block will output all MIDI messages read from the file that have a timestamp less or equal to 40ms (assuming that the timestamps in the file start from zero). At the second activation all messages with timestamp in the range from 41 to 80ms will be sent out, and so on. This will happen regardless of the effective gap that elapses between two consecutive activations. In fact, in general, this gap will not be of exactly 40ms, due to the unpredictable latencies caused by the operating system (Win32 systems are not real time systems!). Anyway, the assumption that this interval is indeed correct, will simplify the development of mixed audio/MIDI performances. Finally, the second output (End of file) is used to signal that the end of the file as been reached. A bang is sent out through this output together with the last output buffer.

2.2 BufferInput

CLSID: 0xd43be306,0xa3c5,0x471e,{0x9a,0xe8,0xe7,0x38,0x1c,0x99,0x37,0x92}

Inputs

SyncIn Any

Outputs

MIDI Commands buffer Midi.MidiBuffer

Parameters

Device Combo

Remarks

This block provides a bufferized input of MIDI commands from the specified Device. The Device parameter is selected among the available devices. The number of alternatives depends on the hardware installed in the computer; a MIDI input is usually available for each sound card. The SyncIn input is the clock of this block. It allows any signal to be used as a clock, i.e., the output of any other block may be connected to this input. The clock signal is used to determine the starting time of a new buffer of MIDI commands; thus, at each clock input, the stored queue of commands is sent out as output, and a new empty queue is set up. This queue will accumulate all the incoming MIDI commands until the next clock signal occurs. Currently, the only block using the buffer provided as output is the BufferDescription block.

2.3 Input

CLSID: 0xC04A0401,0x439B,0x475A,{0x95,0x39,0xD9,0x22,0x4E,0x13,0x46,0x8A}

Outputs

MIDI Message Math.Scalar

Parameters

Device Combo MsgType Combo Channel Integer [1-16] Control Integer [0-127] Filter duplicates CheckBox

Remarks

Receives input data from the specified MIDI device. Only the Byte2 field of the received data is propagated, as output, to the other blocks. The other fields are used just to filter out the undesired messages. The output of this block is an integer scalar number, where only the less significant byte is used.


The Device parameter is used to select among the available devices. The number of alternatives depends on the hardware installed in the computer; a MIDI input is usually available for each sound card. MsgType lets the user select the type of messages that will be accepted by this block; the possible values are Note On, Note Off, Control Change, Key After Touch, Chan After Touch, Pitch Bend, Program Change. Channel specifies the observed channel: note that only data coming from the specified channel is considered, all other channels are simply ignored. The Control parameter acts as a filter based on the value of the Byte1 field of the MIDI message: commands with this value as Byte1 are passed on, all others are filtered out. Finally, Filter duplicates, if enabled, causes only the first instance of consecutive identical messages to pass, and discards all subsequent instances, thus, only the variations of the MIDI observed data are sent out. It is noteworthy that more that one instance of this type of block can be put in a patch, hence, different channels, for instance, can be monitored by means of different instances of this type of block. An alternative is to use the MessageInput block, that does not filter the incoming messages.

2.4 MessageInput

CLSID : 0x743c46e1,0x132c,0x4452,{0xa8,0x6d,0xad,0xcf,0xe3,0x28,0x54,0x99}

Outputs


Parameters

Device Combo

Remarks

Retrieves a MIDI message from the specified device. The output of the block is a MIDI message, i.e., a structure containing the whole data received from the input device.

3 Output

3.1 MessageOutput

CLSID : 0x5047c6fb,0x2167,0x44d9,{0xbe,0xc5,0x46,0x81,0x46,0x86,0xce,0xda}

Inputs


Parameters

Device Combo

Remarks

Sends a MIDI message to the specified device. The input of the block is a MIDI message.

3.2 Output

CLSID : 0xAA6774C6,0xFA74,0x444D,{0xB2,0x9D,0xFD,0x1E,0xF5,0x03,0x7A,0x44}

Inputs

EYWIN_MIDIMSG MathDatatype.Scalar


Parameters

Device Combo MsgType Combo Channel Integer [1,16] Control Integer [0,127] OutOfRangeMode Combo OutOfRangeWarning Boolean Filter duplicates Boolean Disable output Command

Remarks

Sends the input value to the specified output MIDI device. Only the least significant byte of the input integer number is considered. The MIDI message that is output is composed of the input value as Byte2 of the MIDI protocol. The other fields are obtained from the value of the parameters. MsgType can assume the values: Control Change, Note On, Note Off, Program Change, Key After Touch, Chan After Touch, and Pitch Bend. Together with the Channel parameter, they build the Status byte of the MIDI message. The Control parameter determines the value of the Byte2 of the MIDI message. OutOfRangeMode determines the behaviour of the block when a value not included in the range [0,127] is received as input: if Saturate is chosen, then the MIDI message is sent, and the Byte2 is obtained by saturating the input value (i.e., if the input is greater than 127, Byte2 is set to 127; if the input is less than 0, Byte2 is 0); if Stop is chosen, no message is generated; finally, if Last is used, the last valid (in range) input is used. The OutOfRangeWarning parameter, when set to true (checked) enables the block to write a message in the EyesWeb message bar when the input is out of range. Filter duplicates, if enabled, causes only the first instance of consecutive identical messages to pass, and discards all subsequent instances, thus, only the variations of the MIDI observed data are sent out. Finally, the Disable output command toggles the state of the block to enabled or disabled. When the state is disabled (a red cross appears on the block) no message is generated, disregarding the input. Note that this is an old block (implemented prior to version 2.3), and hence it has this command to enable or disable it. Since version 2.3 of EyesWeb this feature can be automatically managed by EyesWeb; as a consequence, this block can be disabled in two different ways, one managed by the block itself through the Disable Output parameter, the other managed by EyesWeb through the Deactivate extended parameter.

4 Operations

4.1 BufferDescription CLSID: 0x18be0488,0x603a,0x4753,{0x99,0x36,0x62,0xa9,0xad,0x72,0x8e,0x6b}

Inputs

MIDICommandsBuffer MIDI.MIDIBuffer

Outputs

Buffer length MathDatatype.Scalar

Parameters

Describe in MessageBar Boolean

Remarks

Provides a description of the content of the input MIDI buffer. The output of the block is an integer scalar number containing the length of the buffer. If the Describe in MessageBar parameter is enabled a description of the content of each command of the input buffer is written in the EyesWeb MessageBar. The format used is the following: (Message) MIDIMSG: N commands (Message) MIDICMD 0: <<ts>>ms 0x<<sb>> 0x<<b1>> 0x<<b2>> (Message) MIDICMD 1: <<ts>>ms 0x<<sb>> 0x<<b1>> 0x<<b2>> ……


(Message) MIDICMD N-1: <<ts>>ms 0x<<sb>> 0x<<b1>> 0x<<b2>> where N is the number of MIDI messages in the input MIDI buffer, <<ts>> represents the timestamp (in milliseconds), <<sb>> is the status byte, <<b1>> is byte1, and <<b2>> is byte2.

4.2 MessageCheckDuplicates

CLSID: 0x6f3602b3,0x8a85,0x467a,{0xb8,0x23,0x75,0x6b,0x62,0x7c,0xd2,0xe5}

Inputs

EYWIN_MIDIMSG MIDI.MIDIMessage

Outputs

EYWOUT_MESSAGE MIDI.MIDIMessage

Parameters

Channel Combo Message Type Combo Byte1 Combo Byte2 Combo Filter Mode Combo

Remarks

This block checks for consecutive identical MIDI messages, and keeps or removes them, according to the value of the parameters. In the normal operating mode (i.e., using the default values for all parameters) two MIDI messages are considered identical if all their fields (Status byte, Byte1, Byte2) are identical: in this case only the first one only is kept while all remaining ones are removed. For every possible field (the status byte is split into the two more meaningful Channel and Message type fields) a combo is available where the values have the following meaning:

Use The field is used when comparing the MIDI messages. Ignore The field is ignored when comparing the MIDI messages, i.e., the messages are

considered identical even if this field differs Exclusive use The field is the only one used for the comparison.

. If all fields are set to Ignore, every message is considered identical to the previous one, if existent. Note that the parameters are automatically maintained in a consistent state, hence, if one field is set to Exclusive, all others are automatically set to Ignore; moreover, if a field is set to Use and there exists one field set to Exclusive, it is automatically set to Use. Once established if the actual message differs from the previous one, it may be sent out or not, according to the value of the Filter Mode parameter:

Remove duplicates The actual message is not sent out, if it turns out to be identical to the previous one. Duplicates Only The actual message is not sent out unless it is identical to the previous one. Remove all All messages are removed, regardless of the difference with the preceding ones. Remove nothing All messages are kept, regardless of the difference with the preceding ones.

4.3 MessageField

CLSID: 0x7962b28d,0xb25e,0x4468,{0x9c,0x8f,0x1e,0xaf,0x76,0x08,0x07,0xfc}

Inputs


Outputs

EYWOUT_FIELD MathDatatype.Scalar


Parameters

Message field Combo

Remarks

Extracts the specified field from the input MIDI message. The output is a scalar integer number, where only the less significant byte is used (the other bytes are zero). The number represents the value of the field extracted from the MIDI message. The input is a MIDI message, that can be provided, for instance, from the MessageInput block. The Message field parameter specifies which field to extract from the MIDI message. The first three alternatives refer to the corresponding fields in the MIDI message, while Channel specifies the four least significant bits of the Status byte and MsgType refers to the four most significant bits.

4.4 MessageFilter

CLSID: 0xc34fac5f,0x37c3,0x4110,{0xa9,0xb1,0x05,0x68,0x77,0x5c,0x05,0xd9}

Inputs


Outputs

EYWOUT_MESSAGE MIDI.MIDIMessage

Parameters

Channel Combo Channel Val Integer [1-16] or RTFunction Integer[0-15] Message Type Combo MsgType Val Combo Byte1 Combo Byte1 Val Integer [0,127] Byte2 Combo Byte2 Val Integer [0,127] Filter Mode Combo

Remarks

This block lets the user filter out some messages or keep only a specified type of message, based on the content of the message itself. For all fields of the MIDI message, a combo specifies whether this field is used in the filtering operation. Possible values are

Use The field is used when filtering the MIDI messages. Ignore The field is ignored when filtering the MIDI messages. Exclusive use The field is the only one used.

The edit box following the combo box determines the value that the field must assume to match the given condition: for instance, if the value of the Channel Val parameter is 3, and if the channel is used (set to Use or Exclusive), then all input messages having 3 as the value of the MIDI channel match the condition. The Filter Mode parameter determines if the messages that match the condifition have to be filtered out, or are the only ones allowed to pass through the filter. If the selected mode is Filter matching then messages that match the condition are filtered out; Pass matching has the opposite meaning, i.e., only messages that match the conditions are allowed to pass. Finally, Filter all and Filter nothing respectively filter out all messages or none of them.

4.5 MessageGenerator - MessageGeneratorSyncIn

CLSID : 0xa28de63c,0xd206,0x40b2,{0x85,0xb0,0x0e,0x7e,0xfb,0x61,0x79,0x01}

CLSID_SyncIn: 0x05362ba4,0x528d,0x42b2,{0xac,0xf6,0xb4,0x81,0x64,0xcc,0xb8,0x61}


Inputs

SyncIn Any (MessageGeneratorSyncIn only)

Outputs


Parameters

Period Integer [0-∞) (MessageGenerator only) Message type Combo Channel Integer [1-16] or RTFunction Integer[0-15] Byte1 Integer [0-127] Byte2 Integer [0-127]

Remarks

Generates a MIDI message. The content of the message is specified by the Message type, Channel, Byte1, and Byte2 parameters. There are two versions of this block: MessageGenerator, which generates the MIDI message periodically, with a period specified by the Period parameter (in milliseconds), and MessageGeneratorSyncIn, which generates the MIDI message using the input as a clock source. Any type of data can be used as input. Note that the content of the input is not important, just its timing is. In other words, a message is generated each time a new input is present, regardless of the value of the input.

Finally, note that if Real time message is chosen as message type, the Channel parameter is interpreted as the real time function to be generated, and its range is [0-15] instead of [1-16].

4.6 NoteOn2NoteOff

CLSID: 0x0e6920c7,0xf4d2,0x450e,{0x90,0x5f,0xff,0xad,0x82,0x08,0x0d,0x23}

Inputs


Outputs

EYWOUT_MIDIMSG MIDI.MIDIMessage

Parameters

Convert2NoteOff Boolean

Remarks

Converts all input NOTE_ON messages to NOTE_OFF if the velocity is zero. All other messages are left unchanged. If the Convert2NoteOff parameter is false (i.e., unchecked), then even the NOTE_ON messages with velocity zero are left unchanged (the block does nothing).

5 Datatypes This part of the guide describes the programming interface of the MIDI library, and can therefore be skipped by users who do not plan to develop their own blocks. The MIDI library comes with two different datatypes, one to handle a single MIDI message, the other to handle buffers of MIDI messages. To use the MIDI datatypes you must include EyesWebSound.h from within your files (you can include it in your stdafx.h, to avoid including it in all files). Moreover, the definition of the CLSIDs and IIDs of the MIDI datatypes could turn out to be useful. The available CLSIDs are: const IID IID_IDTMIDIBuffer =

{0xd9e164ec,0x5535,0x4217,{0xb3,0xe4,0xe4,0x02,0xff,0x51,0x57,0x8a}};

const CLSID CLSID_DTMIDIBuffer =

{0x790a994a,0xf9bb,0x4c5f,{0x98,0x4d,0x2f,0x82,0xf0,0x15,0x6c,0xa1}};

EyesWeb Clock Library Quick Reference 1

Clock Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the License provided.









Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 Clock Library.......................................................................................................................................................... 3

1. Demultiplier................................................................................................................................................ 3 2. ExtractFromSoundStream........................................................................................................................... 3 3. Generator .................................................................................................................................................... 3 4. GeneratorSync ............................................................................................................................................ 3 5. Multiplier .................................................................................................................................................... 3

Datatypes ................................................................................................................................................................ 4 1. PeriodicClock ............................................................................................................................................. 4

References .............................................................................................................................................................. 5


Clock Library The Clock Library is contained in the EywClock.dll DLL file. It contains blocks for constant-period clock handling.

1. Demultiplier Decreases the input clock by a constant integer factor. If the factor is set to 1 the output clock is the same as the input clock. Index selects which clock in the period to use for output; it actually represents the clock phase with respect to the input. Parameters

Factor integer decrease factor Index integer clock phase

2. ExtractFromSoundStream Extracts the clock signal from an audio stream. The input is an audio stream, the output is the associated clock which, in turn, can be used as an input to blocks that need an external clock source.

3. Generator Generates a clock signal using Windows’ Multimedia Timer.

4. GeneratorSync CLSID: 0x71d3e57b,0x820d,0x4b97,{0xa5,0x5f,0x37,0xdf,0xed,0xc0,0x43,0x13}

Inputs

Input clock Clock.PeriodicClock

Outputs

Output clock Clock.PeriodicClock

Parameters

Mode Combo Factor Period

Factor Double

Period Integer

Remarks

This block produces as output a clock signal, using the input one as reference. The Mode parameter determines how the period of the output clock is computed. If Mode is set to Factor, then the period of the generated clock signal is Factor times the period of the input clock. Otherwise, the period is specified (in milliseconds) by the Period parameter.

5. Multiplier Increases the input clock by a constant integer factor.


Parameters

Factor integer increase factor

Datatypes This part of the guide describes the programming interface of the Clock library, and can therefore be skipped by users who do not plan to develop their own blocks. The Clock library comes with one datatype, that is named PeriodicClock. To use the PeriodicClock datatype you must include EyesWebClock.h from within your files (you can include it in your stdafx.h, to avoid including it in all files). Moreover, the definition of the CLSIDs and IIDs of the PeriodicClock datatype could turn out to be useful. The available CLSIDs are: const CLSID CLSID_DTPeriodicClock =

{0xc2b57388,0x3225,0x4979,{0xaf,0xd6,0xea,0xa4,0x51,0x7c,0xce,0x54}};

const IID IID_IDTPeriodicClock =

{0x3ac13f32,0xf7b4,0x430b,{0xa0,0xa8,0xcb,0x8d,0x48,0xef,0x99,0x6e}};

These declarations can be placed somewhere in your cpp files, e.g., in the FactoryData.cpp file. They can be used instead of the string identifiers placed by the Wizard in the template code of your blocks. For instance, when a new block is generated by the Wizard using the PeriodicClock datatype, some piece of code like the following is generated:

SetInputInfo( INPUT_INDEX, "input_name",

"{c2b57388-3225-4979-afd6-eaa4517cce54}",

"{3ac13f32-f7b4-430b-a0a8-cb8d48ef996e}",

EYWIF_REQUIRED_INITIALIZATION | EYWIF_REQUIRED_EXECUTION );

where the two string identifiers are, respectively, the CLSID and the IID of the PeriodicClock datatype. The two identifiers could be replaced manually by the developer with the previously declared variable, in order to obtain the more readable code:

SetInputInfo( INPUT_INDEX, "input_name",

CLSID_DTPeriodicClock,

IID_IDTPeriodicClock,

EYWIF_REQUIRED_INITIALIZATION | EYWIF_REQUIRED_EXECUTION );

However, the proposed operation is just for the sake of readability of the code, and is not necessary for the proper working of the block. Moreover, the same operation can be done in an similar manner for the outputs of the block (the example refers to an input). More generally, the same operations could be performed whenever a CLSID or IID of a datatype is used for developing EyesWeb blocks. The datatypes that are described in the following sections provide a custom COM interface that inherits from the IEyesWebDatatype interface, which on its own, inherits from IUnknown. Only the custom interfaces are described in this guide, as the IEyesWebDatatype interface is described in the EyesWeb Programmers Guide, whereas the IUnkown interface is described in the Microsoft MSDN™ library (see [1]), in every COM-related book, and in several other sources.

1. PeriodicClock CLSID : 0xc2b57388,0x3225,0x4979,{0xaf,0xd6,0xea,0xa4,0x51,0x7c,0xce,0x54};


typedef struct

{

EyesWeb Base Libraries Quick Reference 1

Base Libraries Quick Reference These libraries are part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the License provided.









Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 Generic Library....................................................................................................................................................... 3

1. BangGenerator............................................................................................................................................ 3 2. Decoupler.................................................................................................................................................... 3 3. InputSwitchXto1......................................................................................................................................... 3 4. InputSwitchXto1Sync................................................................................................................................. 3 5. Inverter ....................................................................................................................................................... 4 6. OutputSwitch1toX ...................................................................................................................................... 4 7. Queue.......................................................................................................................................................... 4 8. Snapshot ..................................................................................................................................................... 4 9. Switch ......................................................................................................................................................... 4 10. TimeDelay.................................................................................................................................................. 5 11. TimingInfo ................................................................................................................................................. 5 12. Trigger........................................................................................................................................................ 5

String Library ......................................................................................................................................................... 5 1. Display........................................................................................................................................................ 5 2. FromFile/FromFileSync ............................................................................................................................. 6 3. Generator/GeneratorSync ........................................................................................................................... 6 4. LogToFile ................................................................................................................................................... 7

Network Library ..................................................................................................................................................... 7 1. ReceiveFromNetwork................................................................................................................................. 7 2. SendToNetwork.......................................................................................................................................... 7 Examples ............................................................................................................................................................ 7 Block internals.................................................................................................................................................... 8

Datatypes .............................................................................................................................................................. 11 1. Bang.......................................................................................................................................................... 11 2. String ........................................................................................................................................................ 12

References ............................................................................................................................................................ 12


Generic Library The Generic Library is contained in the EywGeneric.dll and EywGenericDatatype.dll DLL files.

1. BangGenerator CLSID: 0x8a20848a,0x26b2,0x43a1,{0x96,0x41,0x75,0x57,0x6f,0x64,0x2f,0xa7}

Outputs

Bang Generic.Bang

Parameters

Bang Command

Remarks

Generates an output whenever it receives a command. It can be useful to convert command parameters to a valid EyesWeb datatype: command parameters, such as those generated by EyesWeb buttons, cannot be directly connected to the input of a block, even if that block accepts any type of data on that input. Because this block generates a valid EyesWeb datatype as output, its output can be connected to any block that accept that type of data; blocks accepting Any data as input, for instance, will accept this datatype. In most cases, this block can be useful for providing an event-based clock.

2. Decoupler Parameters

Can lose data true | false enable/disable unsampled data storage to prevent data loss

Remarks

Outputs a copy of the input data. Being an active block (as opposed to the Generic.Copier block) its operation may support a different (reduced) timing from the actual input data stream, useful, as an example, before CPU-intensive blocks: in this case the execution speed of the rest of the patch remains unaffected by the "heavy" block (some tweaking of the schedulers’ priority is needed, though). The Can lose data parameter is used when the following scheduler is slower than the preceding one, from which the data stream is flowing. If checked, the following scheduler just samples its input; if unchecked, the input stream is stored in an internal queue to prevent data loss. This setting should be used only when the slower scheduler is able to catch up sooner or later without overflowing.

3. InputSwitchXto1 Parameters

Sel integer zero-based input ID

Remarks

Outputs data entering only from the input specified by the Sel parameter. It is an active block, therefore input data may come from different schedulers (otherwise inputs other than the first would be sampled only when something were available on the first input).

4. InputSwitchXto1Sync


Parameters

Sel integer zero-based input ID

Remarks

Outputs data entering only from the input specified by the Sel parameter. It is a passive block, therefore it should be used only when the different inputs are all connected to the same scheduler.

5. Inverter Parameters

Invert true | false enable/disable channel swap

Remarks

Inverts the data flow between two channels. If Invert is checked (default), the two streams are swapped, otherwise no action is taken (as if the block was not there).

6. OutputSwitch1toX Parameters

Selector integer zero-based output ID

Remarks

Input data is transferred to the output specified by the Selector parameter.

7. Queue Parameters

Length integer delay duration in frames

Remarks

Delays input data by a number of frames specified by the Length parameter: if set to 0, no delay is introduced; if set to x, the n-th input frame is stored in an internal queue and output only when the (n + x)-th frame is available for input.

8. Snapshot Remarks

Stores input data upon receiving the Load command and outputs it whenever new data is read from input. Note that the output is the actual data stored, not data read from input. Upon startup, before any Load occurs, the buffer is initialized to the default value, depending on the datatype (e.g. black for images, silence for audio, 0 for numeric values).

9. Switch Parameters

Mode switch | trigger switch mode


Remarks

Opens/closes a data stream. Its state toggles upon receiving the Switch command when Mode is set to Switch (default). When set to Trigger the stream is opened only for the following input and then it is automatically closed.

10. TimeDelay Parameters

Delay integer delay duration in milliseconds

Remarks

Delays input data by the specified number of milliseconds.

11. TimingInfo Parameters

DeltaT integer time interval (in ms) between two consecutive inputs TimeStamp integer input timestamp Frames per second integer number of frames per second, i.e., 1 / DeltaT Counter integer number of input data received

Remarks

Outputs timing information about its input. The information retrieved depends on the Type parameter, and is output as a scalar (Math library) that may be displayed by a Math.Scalar.Output.Display block.

12. Trigger CLSID: 0xF7F55B0C,0xEDB0,0x4410,{0xBC,0xCE,0x0C,0xDC,0xB0,0x94,0xD6,0xEE}

Inputs

Input Any

Outputs

Output Any

Parameters

Pulse Command

Remarks

Propagates directly the input to the output upon receiving a Pulse command. Note that it is very similar to the Switch block when set to trigger mode; however, this block doesn’t have an internal state that can be saved with the patch. In some cases this may be preferrable, since it doesn’t get the patch marked as saved, and hence it avoids the risk to save an unwanted configuration.

String Library

1. Display CLSID: 0x4DDA3E37,0xFCAF,0x4734,{0xAC,0x7A,0x71,0x2C,0x18,0xDE,0xA0,0x6B}

Inputs


String Generic.String

Parameters

MaxLines Integer TextFont Command BgndColor Command Clear Text Command

Remarks

Displays the incoming strings of text in a popup window. The maximum number of lines are specified through the parameter MaxLines; after this limit has been reached, older lines are forgotten when new ones are received. The appearance of the text can be modified by means of the TextFont command, to specify the font type, colour, and size. The BgndColor parameter chooses a different colour for the background of the window. Finally, the Clear text command forgets all currently stored text.

2. FromFile/FromFileSync CLSID: 0x810b406a,0x36a0,0x49bf,{0x95,0x69,0xbb,0x6b,0x8e,0x26,0xe9,0xd5}

CLSID: 0xaff9d58c,0xf9e4,0x439c,{0x8c,0x25,0x3d,0xa5,0x3e,0x09,0xcf,0x46}

Inputs

SyncIn Any [StringFromFileSync only]

Outputs


Parameters

Period Integer [StringFromFile only] Filename Filename Loop Boolean

Remarks

Reads lines from the input file and outputs them as strings. If the Loop parameter is true, the file is rewound whenever the end of file is reached, otherwise, no more output is generated in that case. There are two versions of this block, one with an internal clock and another accepting a SyncIn input, i.e., an external clock. The first outputs strings with the timing specified by the Period parameter (in milliseconds), the other produces an output whenever an input (of any type: it is only used as clock, thus its content is ignored) is received.

3. Generator/GeneratorSync CLSID: 0x4657bb23,0x9910,0x4a15,{0xb9,0xc4,0x55,0x3f,0x92,0x8b,0x4c,0x32}

CLSID: 0x648aa7d4,0x9bb3,0x4a96,{0x87,0x1f,0x3d,0x97,0x43,0xf5,0x72,0xc3}

Inputs

SyncIn Any [StringGeneratorSync only]

Outputs


Parameters

Period Integer [StringGenerator only] String String

Remarks

Generates a String output, with the value specified as the String parameter. There are two versions of this block, one producing the output periodically, with the period specified by the first parameter, and the other one producing the output only in response to an input of any type.


4. LogToFile CLSID: 0xaa395fd5,0xf83c,0x4eae,{0x93,0x09,0xad,0xc4,0x39,0xe2,0xe6,0xf7}

Inputs


Parameters

Filename Filename Append Boolean

Remarks

Saves the incoming strings to the specified file. If Append is true, and the file already exists, the strings are appended to the end of the file, in all other cases a new file is created.

Network Library EyesWeb supports communications among different computer (potentially running different operating systems) through the TCP/IP Network module, packaged as the EywNetwork.dll. This DLL implements 2 modules, SendToNetwork (found in the Input.Network and Network.Input folders) and ReceiveFromNetwork (found in the Output.Network and Network.Output folders), that can be used to establish a communication channel between two EyesWeb patches running on two computers (N.B. for testing purposes the 2 blocks can be placed in the same patch, simulating a networked patch on a single computer).

1. ReceiveFromNetwork Remarks

Receives data through a TCP/IP connection. This block has an input that optionally can be used to connect it to a SendToNetwork block to explicit the network link existing between the two modules; the type of the output is defined at runtime, and will be the same datatype that has been fed to the corresponding SendToNetwork module. The only parameter accepted by this module is the IP Port where the block will be listening for connections.

2. SendToNetwork Remarks

Sends data through a TCP/IP connection.This block has an input that accepts any valid EyesWeb datatype (image, matrix, scalar, etc…); the output is present only to allow the user to draw the link between this network output module and the corresponding network input module, if present, to help the scheduler to correctly determine the initialization order. The parameters accepted by this module are the target Computer Name (either its dotted IP address or a string recognized by the current DNS or WINS server) and the IP Port (that must be the same specified in the ReceiveToNetwork block that is supposed to receive the message).

Examples In the following example, the output of the processing from Computer A is sent to Computer B, where it is displayed:


Computer A

Computer B

In the following example, Computer A sends some data to Computer B, where it is processed, and the result is sent back to Computer A for displaying:

Computer A

Computer B

Block internals This appendix describes the communication protocol between the two modules, to allow users to write custom code to replace one of the two modules (e.g. if the communication is not intended to be between two EyesWeb patches but between a custom application – even on a non-Windows computer, provided it does support TCP/IP - and EyesWeb) Note: communication is one-way, from the Sender to the Receiver. 1) Datatype initialization: the first information being transmitted by the Sender is the CLSID of the datatype

specified in input. For non-Windows platforms, a CLSID is defined as a 16 byte identifier typedef struct _GUID { unsigned long Data1;

EyesWeb Serial Library Quick Reference 1

Serial Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the License provided.









Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 Serial Library.......................................................................................................................................................... 3

1. Input............................................................................................................................................................ 3 2. Output ......................................................................................................................................................... 3

References .............................................................................................................................................................. 4


Serial Library The Serial Library is contained in the EywSerial.dll and requires the EywMathDatatype.dll DLL file for its runtime usage.

1. Input CLSID : 0x3E54FACE,0x002F,0x4A73,{0x9B,0x8D,0x86,0x9B,0xD4,0x45,0x58,0xEA}

Outputs

CHARACTER Math.Scalar

Parameters

Device Integer [1..9] Mode String Buffer size [unused]

Remarks

Retrieves a character from the specified input serial port. Ports from 1 to 9 can be used. The port is configured via the Mode parameter, which uses the same syntax as the mode system command. See BuildCommDCB in reference [1] for more details.

The output of this block is a 32bit integer number (Math.Scalar), but only the least significant byte is used: the value of the other 3 bytes is left undefined.

It is noteworthy that it is possible to have more than one block working on the same physical device: in this case, the Mode parameter must be compatible among blocks working on the same device; it is even possible to have output blocks sharing the same device: in this case, too, their Mode parameters must be compatible.

2. Output CLSID : 0x63beadea,0x8c26,0x497d,{0x9b,0xb3,0xb7,0x73,0xeb,0x42,0xe0,0xb5}

Inputs

CHARACTER Math.Scalar

Parameters

Device Integer [1..9] Mode String Interbyte delay Non-negative integer Max queue length Non-negative integer

Remarks

Sends a character to the specified output serial port. Ports from 1 to 9 can be used. The port is configured via the Mode string, whicg uses the same syntax as the mode system command. See BuildCommDCB in reference [1] for more details. The Max queue length parameter specifies the number of bytes that can be stored internally by the block, when data is received at a speed greater than that accepted by the serial port. A value of 0 means no limit, any positive value is used as an upper bound. Note that the value does not correspond to the amount of memory actually used by the block, but only to the number of stored characters. Since each character is stored together with more info, the actually used memory is a multiple of this value. Moreover, the actual queue length is the maximum of those specified by the various blocks, when there is more than one block using the same output device.

The parameter Interbyte delay specifies the delay, in milliseconds, between two consecutive sent characters. When this value is greater than zero, characters are sent out guaranteeing that the minimum delay is that specified. When more blocks use the same physical device, the value actually used is the maximum among all blocks.

More blocks can use the same device as input or output; see Input for details.


Similarly to the Input block, the Output block uses only the least significant byte of the 32bit integer number received as input.

References

[1] Microsoft MSDN™ Library: http://msdn.microsoft.com

EyesWeb Math Library Add-On Reference 1

Math Library Quick Reference This library is part of the EyesWeb software environment and can only be distributed with the complete package and used in accordance with the Licence provided.









Table of Contents Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 Math Library........................................................................................................................................................... 3

1. AlphaFilter.................................................................................................................................................. 3 2. ExtractWithIndex........................................................................................................................................ 3 3. MaxMin ...................................................................................................................................................... 4 4. VectorDistance ........................................................................................................................................... 5


Math Library The Math Library is contained in the EywMath.dll, EywMathDatatype.dll and EywMAthExtras.dll DLL files.

1. AlphaFilter CLSID : 0xA5E18FEF,0x3868,0x4C51,{0xA3,0x41,0x00,0x5E,0xDB,0xC4,0x6B,0x26} Inputs

In MathDatatype.Scalar An input stream of scalars. It must be a stream of floats.

Outputs

Out MathDatatype.Scalar The alpha filtered stream of scalars. It is a stream of floats.

Parameters

Alpha Double [0, 1] The actual value of the α parameter of the filter (see the remarks section). The default value is 0.2.

Remarks

This block performs alpha-filtering on the input stream according to the following formula:

)( 11 −− −+=

iiiiyxyy α

where xi is the current input, yi and yi-1 are respectively the current and previous outputs, and α is the value of the Alpha parameter. Note that if the current input is not valid, accordingly the output will be set as invalid. Error and warning messages

During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The input scalar must be float. The input scalars must be real. If your inputs are integer

numbers, you can use a “DomainConv” block in the EyesWeb Math.Scalar library to convert an integer number in a real.



2. ExtractWithIndex CLSID : 0xB74A0ED6,0x5CCE,0x11D5,{0x90,0xCE,0xAD,0xC3,0xC9,0x51,0x35,0x36} Inputs

Input matrix MathDatatype.Matrix The input matrix from which a row or a column has to be extracted.

Outputs

Output matrix MathDatatype.Matrix The extracted row or column. Note that if a row is extracted a vector having one row and a number of columns equal to the number of columns of the input matrix is returned. If a column is extracted a vector having one column and a number of rows equal to the number of


rows of the input matrix is returned.

Parameters

Index Integer The zero-based index of the row/column that has to be extracted. This index must be in the range [0, Number of rows/columns of the input matrix - 1] The default value is 0.

Mode Combo [Design Time] This parameter is used to determine if a row or a column has to be extracted. Two options are thus available “Extract a row” and “Extract a column”. The default value is “Extract a row”.

Remarks

Extracts and returns as output a row or a column of the input matrix. Note that if one of the inputs is not valid, the output will be set as Invalid. Error and warning messages

During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The current index is out of range. The current index is larger than the detected number of

row/columns of the input matrix. Check the value of the Index parameter and the dimensions of the input matrix. Note that the index should be zero based, i.e., its range is [0, Number of rows/columns of the input matrix - 1].

The combo parameter is out of range. The block received an unexpected value for the “Mode” combo parameter.


execution phase will fail. Warning: The current index is out of range. The current index is larger than the detected number of

row/columns of the input matrix. Check the value of the Index parameter and the dimensions of the input matrix. Note that the index should be zero based, i.e., its range is [0, Number of rows/columns of the input matrix - 1]. The previous valid value of the index parameter will be maintained and the corresponding row/column extracted.

Warning: The combo parameter is out of range. The block received an unexpected value for the “Mode” combo parameter. The output will be set as invalid.

3. MaxMin CLSID : 0xD085D5C8,0xDB30,0x47FC,{0x9B,0x3B,0x4C,0xB2,0xF9,0x3F,0xC1,0x03} Inputs

Input MathDatatype.Scalar An input stream of scalars.

Outputs

Max MathDatatype.Scalar The maximum value found so far in the input stream. Min MathDatatype.Scalar The minimum value found so far in the input stream.

Parameters

Reset Command When a reset command is given, the maximum and


minimum values calculated so far are reset and new maximum and minimum values will be calculated starting from the input following the reset command.

Delay Integer [2, +�� The number of calls (i.e., the number of inputs) that are skipped when the patch starts or after a reset command. The default value is 2.

Timeout Integer [0, +�� The number of calls (i.e., the number of inputs) after which the maximum and minimum values are no more calculated. The default value is 0, i.e., no timeout is expected and the maximum and minimum values are continuously updated.

Remarks

Returns as output the maximum and minimum values found so far in the input streams of scalars. The first output is available after the number of calls (i.e., the number of inputs) specified by the Delay parameter. The Delay parameter can be set high enough to avoid transient effects. Output will be set as invalid during the delay phase. A timeout can also be specified (again as number of calls): after it, the maximum and minimum values are no more calculated and the last valid values are maintained. Timeout should be high enough to allow the input signal to be stabilized. A value of 0 for the Timeout parameter specifies that no timeout is required. When a reset command is given, the maximum and minimum values calculated so far are reset and new maximum and minimum values will be calculated starting from the input following the reset command: again, the first valid output will be obtained after the arrival of a number of inputs as specified by the current value of Delay. The Timeout counter is also reset. Note that if one of the inputs is not valid, it is ignored and the last valid maximum and minimum are returned as output. Error and warning messages

During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. During the execution phase Message Description An error occurred in the locking phase. The block could not lock an input or an output. The


4. VectorDistance CLSID: 0x3FBC48F6,0x4B18,0x11D5,{0x81,0x31,0xE1,0xD6,0x48,0xE2,0xFC,0x6D} Inputs

In1 (Main) MathDatatype.Matrix The first vector involved in the distance operation. In2 MathDatatype.Matrix The second vector involved in the distance operation. Note

that the two vectors must have the same dimensions.

Outputs

Out MathDatatype.Scalar The calculated distance between the input vectors. It’s always a real number.

Parameters

Degree Integer (-�� The degree of the distance that has to be calculated (see the remarks section). The default value is 2 corresponding to the standard Euclidean distance

Remarks

Calculates the distance between two input vectors according to the following formula:


where k is defined by the user through the Degree parameter. k = 2 corresponds to the standard Euclidean distance. If Degree (i.e., k) is zero or less than zero, the �-distance is calculated, that is:

Note that if one of the inputs is not valid, the output will be set as Invalid. Error and warning messages


inputs. The input vectors have different dimensions. The input vector must have the same dimensions (i.e., the

same number of columns if row vectors, or the same number of rows if column vectors). Check the dimensions of your input vectors.

The inputs must be vectors, not matrices. Inputs must be vectors, either row vectors (i.e., 1 x n vectors) or column vectors (i.e., n x 1 vectors). Check your inputs.



1 with ),(

1

1

≥

−= ∑

=

kvuvudkn

i

k

iik

iiivuvud −=∞ max),(

EyesWeb 3.3.0 Libraries Add-ons Reference 1

EyesWeb 3.3.0 Libraries Add-ons Reference The blocks documented here are part of the EyesWeb Generic, Imaging, and Math libraries. These libraries are part of the EyesWeb software environment and can only be used in accordance with the License provided.

Copyright Notice EyesWeb is Copyright © 1999-2003, InfoMus Lab (Laboratorio di Informatica Musicale) - DIST - University of Genoa (http://infomus.dist.unige.it)

License agreement Use of EyesWeb (hereinafter 'SOFTWARE') is contingent on your agreement to the following terms: WARRANTY & USE: DIST - University of Genoa grants you a limited, non-exclusive license to use the SOFTWARE free of charge for ANY purpose, commercial or private, without restriction. DIST – University of Genoa makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. DIST - University of Genoa is not obligated to provide maintenance or updates for the SOFTWARE. DISTRIBUTION: the SOFTWARE, in original version or in part, may be freely distributed, provided that this copyright and permission notice appear on all copies and supporting documentation, and that the DIST - University of Genoa copyright notices are referred in the following ways:

a) DIST - University of Genoa's copyright notice should be included in the documentation, regardless of the media used to supply the documentation, and

b) The DIST - University of Genoa and EyesWeb Logos have to appear on packages and promotional material (DIST - University of Genoa makes the logos available on the EyesWeb's ftp site (ftp://infomus.dist.unige.it), and

c) in the 'about box' of the product, in the case that it is not the EyesWeb about box, DIST – University of Genoa and EyesWeb must be cited in the following manner: "EyesWeb is copyright © InfoMus Lab (Laboratorio di Infomatica Musicale) - DIST - University of Genoa (http://infomus.dist.unige.it)", and

d) any public use of EyesWeb, or of any application based on EyesWeb, must be preliminarily notified to [email protected].

Antonio Camurri DIST - University of Genoa Viale Causa, 13 16145 Genova, Italy E-mail: [email protected] This work is based in part on the Intel IPL Library (Copyright © Intel Corporation) and on the OpenCV library.

http://infomus.dist.unige.it/

ftp://infomus.dist.unige.it/

http://infomus.dist.unige.it/

mailto:[email protected]


Table of Contents EyesWeb 3.3.0 Libraries Add-ons.......................................................................................................................... 1 Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 1 Add-ons to the EyesWeb Generic Library.................................................................................................. 4

1.1 Counter ................................................................................................................................................... 4 2 Add-ons to the EyesWeb Imaging Library ................................................................................................. 7

2.1 BlurWithAnchor ..................................................................................................................................... 7 2.2 CalcBlurParams ...................................................................................................................................... 8 2.3 ComputeHistogram................................................................................................................................. 9 2.4 DrawEllipse .......................................................................................................................................... 11 2.5 DrawPolylines ...................................................................................................................................... 13 2.6 DrawRectangles.................................................................................................................................... 17 2.7 MoveBoundingRect.............................................................................................................................. 19 2.8 MultiClickDisplay ................................................................................................................................ 20

3 Add-ons to the EyesWeb Math Library .................................................................................................... 24 3.1 CrossCorrelation................................................................................................................................... 24 3.2 DotProduct............................................................................................................................................ 25 3.3 FFT ....................................................................................................................................................... 26 3.4 FIRFilter ............................................................................................................................................... 28 3.5 Hysteresis.............................................................................................................................................. 29 3.6 PointInRect ........................................................................................................................................... 31 3.7 ProgressiveVector................................................................................................................................. 32

4 Support to FreeFrame plugins................................................................................................................... 33 4.1 Registering FreeFrame plugins............................................................................................................. 33 4.2 Retrieving FreeFrame plugin information ............................................................................................ 34 4.3 Inputs, output, and parameters .............................................................................................................. 34 4.4 Error and warning messages ................................................................................................................. 36 4.5 A sample plugin.................................................................................................................................... 38

5 Sample patches ......................................................................................................................................... 39


EyesWeb 3.3.0 Libraries Add-ons This document describes a collection of EyesWeb blocks enriching the EyesWeb Generic, Imaging, and Math Libraries. The blocks are included in two dll files: Eyw330AddOns.dll and EywFreeFrame.dll. The add-ons contain blocks and patches devoted to: (i) Drawing geometric shapes on screen: these blocks extend the Imaging.Draw Library and partially

improve existing blocks. Blocks for drawing rectangles, ellipses, and polylines (contours) are provided. (ii) Performing analysis and processing of scalar signals in the frequency domain (e.g., blocks for FFT and

for digital filtering). (iii) Miscellaneous blocks and patches: e.g., a generic counter, a block for hysteresis cycles, blocks for

implementing a blur effect “moving” along time, a block for calculating the histogram on gray scale images, a block for selecting multiple points in a given image by clicking on them etc.

The Imaging blocks are partially based on the Intel IPL Library and on the OpenCV Library. The Math blocks are partially based on the Intel SPL Library. A section is devoted to support of FreeFrame plugins (www.freeframe.org) introduced with EyesWeb 3.3.0. A list of sample patches and a brief description of each of them is also provided.

http://www.freeframe.org/


1 Add-ons to the EyesWeb Generic Library

1.1 Counter CLSID: 0xFF77D57C,0x78CB,0x45D8,{0xA6,0xEC,0x67,0x0E,0x0E,0xAF,0x97,0x38}

The Counter block

Input Name Datatype Description Input Any The activation input. The counter is activated each time

something (i.e., any datatype) arrives in this input. Output Name Datatype Description Count Math.Scalar The current value of the counter. It is an integer or a floating-

point scalar depending on the value of “Output type” parameter.

Parameters Name Type Design/Run time Description Start/Stop Command Design time

Run time When this command is invoked, it starts (if stopped) or stops (if started) the counter. This command is exported by default.

Reset Command Design time Run time

When this command is invoked, the counter isreset, i.e., it is again initialized to the “Beginning”value. This command is exported by default.

Counter type Combo Design time Run time

The kind of counter that should be used. Three options are currently available: “Progressive limited” (the counter stops when it reaches its “End” value), “Progressive unlimited” (the counter continues to increase/decrease indefinitely), “Circular” (when the counter reaches its “End” value, it starts again counting from its “Beginning” value). The default value is “Progressive limited”.

Output type Combo Design time

The type of the output scalar. Two options are currently available: “Integer”, i.e., the output is an integer scalar, and “Real”, i.e., the output is a floating-point scalar. The default value is “Integer”.

Beginning Double (-∞, +∞) Design time Run time

The starting value for the counter. Default value is 0.

End Double (-∞, +∞) Design time Run time

The ending value for the counter. The default value is 10. Notice that this parameter is enabled only if the “Counter type” is not “Progressive unlimited”.

Step Double (-∞, +∞) Design time Run time

The step to be used in the count. A positive step generates an increasing counter, a negative step a decreasing counter. Fractional steps are allowed if the “Output type” is set to “Real”. The default value is 1.

Start automatically after reset

Boolean Design time Run time

If this parameter is set to “True” the counter automatically restart after a reset command is invoked; else a “Start” command must be invoked to restart the counter. The default value is “False”.


Remarks This block implements a counter enabling the user to specify both the kind of counter and the range of the count. Three kinds of counters are currently supported: (i) “Progressive limited”, i.e., the counter stops once it reaches its “End” value. The “End” value continues

to be returned until a new “Start” command is given. (ii) “Progressive unlimited”, i.e., the counter continues its count indefinitely. In this case the “End”

parameter is disabled and the counter is reset only if it reaches one of the limit values in the current implementation, e.g., the maximum integer number that can be represented in the employed system in case the outputs of the counter are integer numbers.

(iii) “Circular”, i.e., when the counter reaches its “End” value, it restarts its count from the “Beginning” value.

The “Counter type” can be changed at run-time. Notice that in case the type is changed from “Progressive unlimited” to “Progressive limited”, if the count already reached the “End” value, the output will be set to the “End” value and the counter will stop. Instead, in case the type is changed from “Progressive unlimited” to “Circular”, if the count already reached the “End” value, the output will be set to the “Beginning” value and the counter will restart its count. The range of the count is specified by the “Beginning” and “End” parameters. These are double precision floating-point numbers in the range allowed by the current system. In case the output of the counter is set to “Integer” and “Beginning” and/or “End” contain a fractional part, such fractional part will be ignored and only the integer part will be considered. The step of the counter is also provided through a double precision floating-point number (the “Step” parameter). Again, if the output is set to “Integer” and “Step” contains a fractional part, such fractional part will be ignored and only the integer part will be considered. If “Step” is a positive number an increasing counter will be obtained; if it is a negative number the counter will be a decreasing counter. In case the values of “Beginning” and “End” and the sign of “Step” are incoherent (e.g., the step is positive, i.e., the counter should increase its count, but “End” is smaller than “Beginning), the sign of “Step” is inverted while “Beginning” and “End” are kept. The counter can be controlled through the “Start/Stop” and “Reset” commands. When the patch is initially locked, the output is set to invalid until a “Start” command is invoked. Once the counter is started, it can be stopped by invoking again the “Start/Stop” command. Thus, repeated “Start/Stop” commands toggle the counter on and off. A “Reset” command reset the counter, i.e., stops the count and reinitialize the counter to its “Beginning” value. If the “Start automatically after reset” parameter is set to “True” the counter automatically restart counting from its “Beginning” value after reset; otherwise the output is set to invalid and a “Start” command is needed to restart the counter. Error and warning messages During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output.

The execution phase will fail. Warning: The counter reached the maximum or minimum allowed value. It will be reset.

The currently supported maximum or minimum integer or double value has been reached. The counter will be reset and it will restart from the indicated “Beginning” value.

In setting the parameters of the block Message Description Warning: The type of the output is integer but the Beginning parameter is float: only the integer part will be considered.

A “Beginning” value having a fractional part has been passed as parameter, but the “Output type” of the counter is set to “Integer”. Only the integer part of the “Beginning” value will be considered.

Warning: The type of the output is integer but the End parameter is float: only the integer part will be considered.

An “End” value having a fractional part has been passed as parameter, but the “Output type” of the counter is set to “Integer”. Only the integer part of the “End” value will be considered.

Warning: The type of the output is integer but the Step parameter is float: only the integer part will be considered.

A “Step” value having a fractional part has been passed as parameter, but the “Output type” of the counter is set to “Integer”. Only the integer part of the “Step” value


will be considered. Warning: Step is positive but End < Beginning: the sign of Step will be changed.

The “Step” value is positive, i.e., an increasing counter has been selected, but “End” is smaller than “Beginning”. The sign of “Step” will be changed, i.e., a decreasing counter having the same step magnitude will be implemented.

Warning: Step is negative but End > Beginning: the sign of Step will be changed.

The “Step” value is negative, i.e., a decreasing counter has been selected, but “End” is larger than “Beginning”. The sign of “Step” will be changed, i.e., an increasing counter having the same step magnitude will be implemented.


2 Add-ons to the EyesWeb Imaging Library

2.1 BlurWithAnchor CLSID: 0xE59401E6,0x5AC4,0x48EF,{0x81,0xA4,0xC3,0x8E,0xE4,0xD4,0x5E,0x58}

The BlurWithAnchor block

Input Name Datatype Description Input image Imaging.Image The input image on which the blur filter should be applied. Output Name Datatype Description Output image Imaging.Image The blurred image. It has the same dimensions and properties

of the input image. Parameters Name Type Design/Run time Description Number of rows

Integer [1, +∞) Design time Run time

Number of rows in the neighborhood matrix to be used by the averaging filter. The default value is 3.

Number of columns


Number of columns in the neighborhood matrix to be used by the averaging filter. Default value is 3.

Anchor x Integer [0, “Number of columns” – 1]

Design time Run time

The x index of the anchor cell of the neighborhood. The default value is 1. See the Remarks section for more details.

Anchor y Integer [0, “Number of rows” – 1]


The y index of the anchor cell of the neighborhood. The default value is 1. See the Remarks section for more details.

Remarks This block extends the features of the EyesWeb LinearFilterBlur block. In particular, it allows the user to specify an anchor point in the neighborhood matrix in order to skew the neighborhood with respect to its geometric center. The anchor point is specified by its x and y indexes in the neighborhood matrix. Thus, in the coordinate system employed by the block, the top left corner would be [0, 0] and the bottom right corner would be [“Number of columns” - 1, “Number of rows” - 1]. For example, for a 3 by 3 neighborhood, the coordinates of the geometric center would be [1, 1]. The block performs the usual blur operation, i.e., it sets each pixel in the output image as the average of all the input image pixels in the neighborhood of size “Number of row” by “Number of columns” with the selected anchor cell. This block is based on the Intel Image Processing Library (IPL). Error and warning messages During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output.

The execution phase will fail. Error in getting the input ipl image. The block could not obtain a pointer to the input or the

output image. The execution phase will fail. In setting the parameters of the block Message Description Warning: The anchor x coordinates must be in the The provided x coordinate of the anchor point is invalid,


range [0, M]. The old value must be kept. i.e., it is negative or larger than “Number of columns” – 1. The old value will be kept. Note that in the warning message “M” will be replaced by the current value of “Number of columns” – 1.

Warning: The anchor y coordinates must be in the range [0, N]. The old value must be kept.

The provided y coordinate of the anchor point is invalid, i.e., it is negative or larger than “Number of rows” – 1. The old value will be kept. Note that in the warning message “N” will be replaced by the current value of “Number of rows” – 1.

2.2 CalcBlurParams CLSID: 0x6FCE3F34,0x0B5F,0x44E4,{0xAC,0x5A,0xEF,0x6B,0x01,0xD2,0x9A,0x7D}

The CalcBlurParams block Inputs Name Datatype Description Radius MathDatatype.Scalar The radius for the “moving blur” effect. See the Remarks

section for more details. Direction MathDatatype.Scalar The direction for the “moving blur” effect. See the Remarks

section for more details. Speed MathDatatype.Scalar The speed for the “moving blur” effect. See the Remarks

section for more details. Outputs Name Datatype Description Number of Rows

MathDatatype.Scalar Number of rows in the neighborhood matrix to be used by the BlurWithAnchor block (see the description of BlurWithAnchor).

Number of columns

MathDatatype.Scalar Number of columns in the neighborhood matrix to be used by the BlurWithAnchor block (see the description of BlurWithAnchor).

Anchor x MathDatatype.Scalar The x index of the anchor cell of the neighborhood to be used by the BlurWithAnchor block (see the description of BlurWithAnchor).

Anchor y MathDatatype.Scalar The y index of the anchor cell of the neighborhood to be used by the BlurWithAnchor block (see the description of BlurWithAnchor).

Parameters This block does not have any parameter. Remarks This block computes the values to be passed as parameters to a BlurWithAnchor block. The block allows implementing the effect of a “moving blur” by specifying as input the radius, direction and speed for such “moving blur” effect. “Radius” should be a positive number; “Direction” should be a number in the range [0, 8] associated to the cardinal points as in the following table:


Direction dirX DirY 0 No direction 0 0 1 N 0 -1 2 NE 1 -1 3 E 1 0 4 SE 1 1 5 S 0 1 6 SW -1 1 7 W -1 0 8 NW -1 -1

“Speed” should be a number in the range [-Radius, Radius], a negative speed meaning an inversion in the direction. The blur parameters are computed as follows: “Number of rows” = “Number of columns” = “Radius” × 2 + 1 “Anchor x” = “Radius” + “Speed” × “dirX” “Anchor y” = “Radius” + “Speed” × “dirY” where “dirX” and “dirY” are the numbers listed in the table above. In case at least one of the parameters is out of range, the block generates the following output: “Number of rows” = 1, “Number of columns” = 1, “Anchor x” = 0, “Anchor y” = 0, i.e., no blur will be applied by BlurWithAnchor block. Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of its inputs. During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output.

The execution phase will fail. Warning: The input radius should have a positive value. No blur will be applied.

A negative value has been received as the radius input. The block will generate the following output: “Number of rows” = 1, “Number of columns” = 1, “Anchor x” = 0, “Anchor y” = 0, i.e., no blur will be applied by BlurWithAnchor.

Warning: Direction must be in the range 0..7 (N..NW). No blur will be applied.

The input value for “Direction” is out of the allowed range [0, 7]. The block will generate the following output: “Number of rows” = 1, “Number of columns” = 1, “Anchor x” = 0, “Anchor y” = 0, i.e., no blur will be applied by BlurWithAnchor.

Warning: Speed must be in the range [-Radius, Radius]. No blur will be applied.

The input value for “Speed” is out of the allowed range [-Radius, Radius]. The block will generate the following output: “Number of rows” = 1, “Number of columns” = 1, “Anchor x” = 0, “Anchor y” = 0, i.e., no blur will be applied by BlurWithAnchor.

2.3 ComputeHistogram CLSID: 0x56D4B245,0x2645,0x4D53,{0x9D,0x16,0xC7,0x4F,0xBC,0xAD,0x0B,0x8F}

The ComputeHistogram block


Input Name Datatype Description Input image Imaging.Image The input image on which the histogram has to be calculated.

It must be a single channel image. Output Name Datatype Description Histogram MathDatatype.Matrix The calculated histogram. It is a vector of floating point

numbers. The number of items is 2d, where d is the depth of the input image.

Parameters Name Type Design/Run time Description Do not clear Boolean Design time

Run time If this parameter is set to “False”, the output histogram is cleared before computing a new one, i.e., a new histogram is calculated for each input frame; otherwise the histogram is simply updated, i.e., the histogram of a new frame is computed on the basis of the histograms of the previous frames. The default value is “False”.

Normalize Boolean Design time Run time

If this parameter is set to “True” the block calculates a normalized histogram, i.e., it scales the histogram such that the sum of its bins becomes equal to “Normalization factor”. Notice that this parameter is enabled only if “Do not clear” is set to “False”. The default value is “False”.

Normalization factor

Double (-∞, +∞)


The normalization factor employed for rescaling a normalized histogram (see above). Notice that this parameter is enabled only if “Do not clear” is set to “False”. The default value is 1.

Row vector Boolean Design time If this parameter is set to “True” the output is a row vector (i.e., a vector having 1 x 2d items); otherwise the output is column vector (i.e., a vector having 2d x 1 items). d is the depth of the input image. Default value is “False”.

Remarks This block calculates and returns as output the histogram of the single channel input image. The output histogram is provided as a vector of 1 x 2d or 2d x 1 items (depending on the value of the “Row vector” parameter) where d is the depth of the input image. For example, in the common case of single channel 8-bit depth images, the output histogram will have 28 = 256 bins. If the “Do not clear” parameter is set to “False”, the output histogram is cleared before computing a new one, i.e., a new histogram is calculated for each input frame; otherwise the histogram is simply updated, i.e., the histogram of a new frame is computed on the basis of the histograms of the previous frames. If the “Normalize” parameter is set to “True” the block calculates a normalized histogram, i.e., it scales the histogram such that the sum of its bins becomes equal to “Normalization factor”. To guarantee output consistency the “Normalize” parameter is considered only if “Do not clear” is set to “False”, otherwise “Normalize” is automatically set to “False” and both the “Normalize” and “Normalization factor” parameters are disabled until “Do not clear” is “True”. This block is based on the Open Computer Vision (OpenCV) library. Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of its inputs. This block works just on single channel images The input image must be a single channel image. Check

the properties of your input image. You can use a conversion block from the EyesWeb Imaging Library to obtain a single channel image.

Error in creating the histogram structure The block failed in creating the histogram internal data


structure. The init phase will fail. During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output.

The execution phase will fail. Error in getting the input ipl image. The block could not obtain a pointer to the input image.

The execution phase will fail.

2.4 DrawEllipse CLSID: 0x7089AF9A,0x6382,0x4959,{0xAC,0xC4,0x7E,0x96,0x5F,0x71,0x79,0xC2}

The DrawEllipse block Inputs Name Datatype Description Input image Imaging.Image The input image. It should be an arbitrary 8-bit image or a single-

channel image with larger depth. Axes MathDatatype.Matrix A 2x1 or 1x2 vector whose items are the lengths in pixels of the

major axis and of the minor axis of the ellipse respectively. Center MathDatatype.Matrix A 2x1 or 1x2 vector whose items are the x and y coordinates in

pixels of the center of the ellipse. Orientation MathDatatype.Scalar The rotation angle of the major axis of the ellipse. Output Name Datatype Description Output image Imaging.Image An image of the same type of the input image, containing the

input image and the drawn ellipse Parameters Name Type Design/Run time Description Thickness Integer [1, +∞) Design time

Run time The thickness in pixels of the border of the ellipse. The default value is 1. This parameter is enabled only if an empty ellipse is drawn (i.e., the “Fill ellipse” parameter is set to “False”).

Fill ellipse Boolean Design time Run time

If this parameter is set to “True” the block draws a filled ellipse with the selected color. Otherwise, an empty ellipse is drawn. Angles are measured in degrees. The default value is “False”.

Start angle Double [0, 360] Design time Run time

This parameter together with the “End angle” parameter allows drawing only part of the input ellipse, i.e., the elliptic arc in the range [“Start angle”, “End angle”]. Angles are measured in degrees. The default value is 0.

End angle Double [0, 360] Design time Run time

This parameter together with the “Start angle” parameter allows drawing only part of the input ellipse, i.e., the elliptic arc in the range [“Start angle”, “End angle”]. The default value is 360.

Color Combo Design time A combo allowing the user to choose the mode of


selection Run time color selection. Two options are currently available: “Selected by users”, i.e., the user can directly select a color by means of a suitable dialog box, and “Controlled within the patch”, i.e., the color is determined by the values of the “Fract R”, “Fract G” and “Fract B” parameters (possibly exported and controlled by other blocks). The default value is “Selected by users”.

Set color Command Design time Run time

When a “Set color” command is invoked a dialog box is open allowing the user to select a color for drawing. This parameter is enabled only if the “Selected by users” color selection mode was selected.

Fract R Double [0, 1] Design time Run time

The R value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. Note that the range of this parameter is [0, 1]. The default value is 1.

Fract G Double [0, 1] Design time Run time

The G value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. Note that the range of this parameter is [0, 1]. The default value is 0.

Fract B Double [0, 1] Design time Run time

The B value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. Note that the range of this parameter is [0, 1]. The default value is 0.

Anti-aliasing Boolean Design time Run time

If set to “True”, it enables drawing of antialiased ellipses. The default value is “False”.

Scaling factor Integer [0, +∞) Design time Run time

This parameter specifies the number of fractional bits in the center coordinates and axes sizes. The default value is 0, corresponding to un-scaled ellipses. Notice that this parameter is enabled only if the “Anti-aliasing” parameter is set to “True”.

Remarks The DrawEllipse block draws an ellipse on the input image. This should be an arbitrary 8-bit image or a single-channel image with larger depth. The output image will have the same dimensions and properties of the input image. The mathematical properties of the drawn ellipse (length of axes, center, orientation) are specified by the block’s inputs. The color used to draw the ellipse can either be directly selected by users or can be determined within the patch by connecting the output of some block to the “Fract R”, “Fract G”, and “Fract B” parameters. Notice that the range of the “Fract R”, “Fract G”, and “Fract B” parameters is [0, 1], 0 meaning complete absence of the corresponding color component and 1 complete presence. Depending on the “Fill ellipse” parameter, the ellipse can be filled or empty. If it is empty, the border thickness can be specified through the “Thickness” parameter. Depending on the “Start angle” and “End angle” parameters only an elliptic arc instead of the whole ellipse will be drawn. In case one of the mathematical inputs is invalid, no ellipse will be drawn (i.e., the output image will be a copy of the input image). Drawing of antialiased ellipses is also supported by means of the “Anti-aliasing” and “Scaling factor” parameters. This block is based on the Open Computer Vision (OpenCV) library. Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of one of its

inputs. Multiple channels images must have 8-bit depth. OpenCV drawing functions work with arbitrary 8-bit

images or single-channel images with larger depth. You can use a conversion block from the EyesWeb Imaging


Library to obtain a compatible image. The lengths of the ellipse axes must be provided as a 2x1 or 1x2 vector.

The lengths of the ellipse axes should be provided as the items of a 2x1 or 1x2 vector. Check the dimensions or your input vector.

The coordinates of the center of the ellipse must be provided as a 2x1 or 1x2 vector.

The x and y coordinates of the center of the ellipse should be provided as the items of a 2x1 or 1x2 vector. Check the dimensions or your input vector.

During the execution phase Message Description An error occurred in the locking phase. The block could lock one of its inputs or the output. The

execution phase will fail. An error occurred while getting the ipl image. The block could not obtain a pointer to the input or the

output image. The execution phase will fail. In managing the block’s private data Message Description An error occurred while saving private data. The block could not correctly save its private data: the

information about the user’s selected color will be lost. An error occurred while loading private data. The block could not correctly load its private data: the

default values will be used for color selection.

2.5 DrawPolylines CLSID: 0x07A48D15,0x9AEE,0x4295,{0xB5,0x7A,0xCD,0x19,0xCD,0x7C,0xB3,0xA5}

The DrawPolylines block Inputs Name Datatype Description Input image Imaging.Image The input image. It should be an arbitrary 8-bit image or a single-

channel image with larger depth. Input points MathDatatype.Matrix An Nx2 matrix containing the x and y coordinates (in pixels) of

the 2D points that are vertices of the polylines that have to be drawn. Each row of the matrix corresponds to the coordinates of a single point.

Number of points for each polyline

MathDatatype.Matrix An Mx1 matrix containing the number of points in the “Input points” matrix belonging to each polyline (for more details see the Remarks section). The sum of the items in this matrix must be equal to the number of rows of the “Input points” matrix. This matrix must be a matrix of integer numbers.

Output Name Datatype Description Output image Imaging.Image An image of the same type of the input image, containing the

input image and the drawn polylines. Parameters Name Type Design/Run time Description Thickness Integer [1, +∞) Design time The thickness in pixels of the lines of the polylines.


Run time The default value is 1. This parameter is enabled only if empty polygons or not closed polylines are drawn (i.e., the “Fill polygon” parameter is set to “False”) and the “Anti-aliasing” parameter is set to “False” as well.

Fill polygons Boolean Design time Run time

If this parameter is set to “True” the block draws filled polygons with the selected color. The borders of the drawn polygons are the (closed) polylines specified in the input matrices. When this parameter is set to “True”, the “Close polylines” parameter is automatically set to “True” and disabled, and the “Anti-aliasing” parameter is automatically set to “False” and disabled. The default value is “False”.

Close polylines


If this parameter is set to “True”, a line is drawn from the last vertex of each polyline to its first vertex, thus closing the polyline and creating a polygon. This parameter is automatically set to “True” and disabled, when the “Fill polygons” parameter is set to “True”. The default value is “False”.

Color selection

Combo Design time Run time

A combo allowing the user to choose the mode of color selection. Two options are currently available: “Selected by users”, i.e., the user can directly select a color by means of a suitable dialog box, and “Controlled within the patch”, i.e., the color is determined by the values of the “Fract R”, “Fract G” and “Fract B” parameters (possibly exported and controlled by other blocks). The default value is “Selected by users”.


When a “Set color” command is invoked a dialog box is opened allowing the user to select a color for drawing. This parameter is enabled only if the “Selected by users” color selection mode was selected.




The G value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. Note that the range of this parameter is [0, 1]. The default value is 0.



Anti-aliasing Boolean Design time Run time

If set to “True”, it enables drawing of antialiased polylines. The default value is “False”.

Scaling factor Integer [0, +∞) Design time Run time

This parameter specifies the number of fractional bits in the coordinates of polyline vertices. The default value is 0, corresponding to un-scaled polylines. Notice that this parameter is enabled only if the “Anti-aliasing” parameter is set to “True”.

Limit input Boolean Design time Run time

If this parameter is set to “True”, the user is enabled to specify how many polylines the block has to draw, i.e., even if the input matrices specify


more polylines only the first ones in the list up to the one decided by the user are drawn. The default value is “False”.

Number of polylines


The number of polylines that have to be drawn (see the “Limit input” parameter). This parameter is enabled only if the “Limit input” parameter is set to “True”. The default value is 1.

Remarks The DrawPolylines block draws one or more polylines (also referenced as contours) on the input image. This should be an arbitrary 8-bit image or a single-channel image with larger depth. The output image will have the same dimensions and properties of the input image. Polylines are provided by means of two matrices. The first one (the “Input points” matrix) is an Nx2 matrix containing in each row the x and y coordinates (in pixels) of the points that are the vertices of the polylines. The second one (the “Number of points for each polyline” matrix) is an Mx1 matrix containing in each row the number of points in the “Input points” matrix belonging to the polyline associated to that row. Thus the number of rows in the “Number of points for each polyline” matrix is equal to the number of polylines that have to be drawn and the sum of the items in the “Number of points for each polyline” matrix must be equal to the number of rows of the “Input points” matrix. For example, consider you want to draw a triangle (vertices: t1, t2, and t3), a pentagon (vertices: p1, p2, p3, p4, and p5), and a rectangle (vertices: r1, r2, r3, and r4). The two input matrices will be built as follows:

=

=453

polyline each for points of Number

rr

rrrp

rrrp

pppp

pppp

ttt

ttt

points Input

yx

y

y

y

y

x

x

x

x

y

y

y

y

x

x

x

x

y

y

y

x

x

x

44

3215

3215

4321

4321

321

321

In case the input points are not enough, i.e., the sum of the items in the “Number of points for each polyline” matrix is larger than the number of rows of the “Input point” matrix, a warning message will be generated and no polyline will be drawn, i.e., the output image will be just a copy of the input image. The block will behave in the same way, in case at least one of the input matrices is invalid. The user can decide to limit the number of polylines, i.e., he/she can decide that some of the polylines in the input matrices do not have to be drawn. In order to do this, the “Limit input” parameter has to be set to “True”. Then the user has to specify how many polylines should be considered through the “Number of polylines” parameter. The block will consider only the given number of polylines starting from the first ones in the input matrices. For example, if the matrices previously discussed are received as inputs and “Number of polylines” is set to 2, the block will draw only the triangle and the pentagon, while the rectangle will be skipped. The block will consider in the “Input points” matrix a number of points equal to the sum of the considered items in “Number of points for each polyline” matrix (e.g., 8 = 3+5 in the example sketched above). If more polylines than the current number of rows of the “Number of points for each polyline” matrix is specified in the “Number of polylines” parameter a warning message is generated and the user-specified limit is ignored. The color employed to draw polylines can either be directly selected by users or it can be determined within the patch by connecting the output of some block to the “Fract R”, “Fract G”, and “Fract B” parameters. Notice that the range of the “Fract R”, “Fract G”, and “Fract B” parameters is [0, 1], 0 meaning complete absence of the corresponding color component and 1 complete presence. Depending on the “Fill polylines” parameter, the polylines can be filled or empty. If they are empty, the line thickness can be specified through the “Thickness” parameter. Depending on the “Close polyline” parameter, polylines can be closed, i.e., the last point is connected with the first point thus generating a polygon, or open. Open polylines cannot be filled, therefore the “Close polyline” parameter is automatically set to “True” and disabled when the “Fill polygons” parameter is set to “True”. Drawing of antialiased polylines is also supported


by means of the “Anti-aliasing” and “Scaling factor” parameters. This block is based on the Open Computer Vision (OpenCV) library. Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of one of its


images or single-channel images with larger depth. You can use a conversion block from the EyesWeb Imaging Library to obtain a compatible image.

The points of the polylines must be provided as an Nx2 matrix.

The points that are the vertices of the input polylines must be provided in an Nx2 matrix where each row corresponds to the x and y coordinates (in pixels) of one point. Check the dimensions or your input matrix.

The number of points in each polyline must be provided as an Mx1 matrix.

The number of points in the “Input points” matrix belonging to each polyline must be provided in an Mx1 matrix where each row corresponds to the number of points in one polyline. Check the dimensions or your input matrix.

The number of points in each polyline must be provided as a matrix of integer numebrs.

The number of points belonging to each polyline must be provided in an Mx1 matrix of integer numbers. Check the type or your input matrix. In case of need, you can use a conversion block from the EyesWeb Math Library to get a matrix of integers numbers.



output image. The execution phase will fail. Warning: Too few points provided: impossible to draw the specified polylines.

The “Input points” matrix contains too few points with respect to what specified in the “Number of points for each polyline” matrix. No polyline will be drawn. The same input image will be returned as output. Check that the sum of the items of the “Number of points for each polyline” is equal to the number of rows of the “Input points” matrix.

Warning: You specified more polylines than currently contained in the input matrices.

“Limit input” is enabled and the value of the “Number of polylines” parameters is larger the number of rows in the “Number of points for each polyline” matrix, i.e., you specified more polylines than those currently available. The user-specified number will be ignored and the block will work as if the “Limit input” parameter were set to “False”.

In managing the block’s private data Message Description An error occurred while saving private data. The block could not correctly save its private data: the




2.6 DrawRectangles CLSID: 0xC83126A8,0xE69D,0x4C37,{0xB4,0x54,0x69,0xFE,0x54,0x28,0xF6,0x3A}

The DrawRectangles block Inputs Name Datatype Description Input image Imaging.Image The input image. It should be an arbitrary 8-bit image or a single-

channel image with larger depth. Input rectangles MathDatatype.Matrix An Nx2 matrix (where N is an even number) containing the x and

y coordinates of the vertices of the rectangles that have to be drawn. A rectangle is specified by providing the x and y coordinates of its top-left and bottom-right corners as the first and second row respectively of the associated matrix. If multiple rectangles are provided the matrices used to represent them are composed by columns in the input matrix, i.e., rows 0 and 1 will provide the first rectangle, rows 2 and 3 the second, rows 4 and 5 the third and so on.

Output Name Datatype Description Output image Imaging.Image An image of the same type of the input image, containing the

input image and the drawn rectangles. Parameters Name Type Design/Run time Description Thickness Integer [1, +∞) Design time

Run time The thickness in pixels of the border of the rectangles. The default value is 1. This parameter is enabled only if empty rectangles are drawn (i.e., the “Fill rectangles” parameter is set to “False”).

Fill rectangles Boolean Design time Run time

If this parameter is set to “True” the block draws filled rectangles with the selected color. Otherwise, empty rectangles are drawn. The default value is “False”.

Color selection


A combo allowing the user to choose the mode of color selection. Two options are currently available: “Selected by users”, i.e., the user can directly select a color by means of a suitable dialog box, and “Controlled within the patch”, i.e., the color is determined by the values of the “Fract R”, “Fract G” and “Fract B” parameters (possibly exported and controlled by other blocks). The default value is “Selected by users”.


When a “Set color” command is invoked a dialog box is opened allowing the user to select a color for drawing. This parameter is enabled only if the “Selected by users” color selection mode was selected.




The G value in the RGB color model. This parameter is enabled only if the “Controlled within


the patch” color selection mode was selected. Note that the range of this parameter is [0, 1]. The default value is 0.



Remarks The DrawRectangles block draws one or more rectangles on the input image. This should be an arbitrary 8-bit image or a single-channel image with larger depth. The output image will have the same dimensions and properties of the input image. Rectangles are provided as an Nx2 matrix where N is an even number. Each rectangle is provided as a 2x2 matrix where the first row contains the x and y coordinates in pixels of its top-left corner and the second row contains the x and y coordinates in pixels of its bottom-right corner. In case multiple rectangles have to be drawn, the matrices representing them must be composed by columns in an Nx2 matrix and provided as input to the block. That is, rows 0 and 1 of the composed matrix will provide the first rectangle, rows 2 and 3 the second, rows 4 and 5 the third and so on. The color used to draw the rectangles can either be directly selected by users or can be determined within the patch by connecting the output of some block to the “Fract R”, “Fract G”, and “Fract B” parameters. Notice that the range of the “Fract R”, “Fract G”, and “Fract B” parameters is [0, 1], 0 meaning complete absence of the corresponding color component and 1 complete presence. Depending on the “Fill rectangles” parameter, the rectangles can be filled or empty. If they are empty, the border thickness can be specified through the “Thickness” parameter. In case the input matrix is invalid, no rectangle will be drawn (i.e., the output image will be a copy of the input image). This block is based on the Open Computer Vision (OpenCV) library. Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of one of its


images or single-channel images with larger depth. You can use a conversion block from the EyesWeb Imaging Library to obtain a compatible image.

The input rectangles must be provided as an Nx2 matrix where N is an even number.

The input rectangles must be provided in an Nx2 matrix (N even number) composed as specified in the remarks section. Check the dimensions or your input matrix.



output image. The execution phase will fail. In managing the block’s private data Message Description An error occurred while saving private data. The block could not correctly save its private data: the




2.7 MoveBoundingRect CLSID: 0x53DDF2AE,0xECC6,0x49BD,{0xA0,0x4D,0x50,0xDB,0x4C,0x7A,0xE3,0x28}

The MoveBoundingRect block

Inputs Name Datatype Description Input image Imaging.Image The input image. A given rectangular region of this image will

be extracted and copied at a given position in the output image. Bounding rectangle

MathDatatype.Matrix The rectangle specifying the region of the input image that has to be extracted and copied in the output image. It is provided as a 2x2 matrix of integer numbers. The first row of such matrix contains the x and y coordinates of the top-left corner of the rectangle; the second row contains the x and y coordinates of the bottom-right corner.

Output Name Datatype Description Output image Imaging.Image The output image. It is a black image (same dimensions and

properties of the input image) on which the specified rectangular region extracted from the input image is copied at the position specified by the block’s parameters.

Parameters Name Type Design/Run time Description Reference point


The point of the extracted rectangle the x and y coordinates of which are specified in the output image coordinate reference system in order to copy the rectangle in the output image. Available options are: “Center”, “Top left”, “Bottom left”. The extracted rectangle will be copied in the output image so that the selected point is at the x and y coordinates specified by the remaining parameters. The default value is “Center”.

Center X or Top or Bottom


The first coordinate (in pixels) in the output image of the point specified as reference for copying the rectangle: “Center X” if “Center” was selected, “Top” if “Top left”, “Bottom” if “Bottom left”. The default value is 176 (half of an image having width = 352 pixels).

Center Y or Left

Integer [0, +∞)


The second coordinate (in pixels) in the output image of the point specified as reference for copying the rectangle: “Center Y” if “Center” was selected, “Left” if “Top left” or “Bottom left”. The default value is 144 (half of an image having height = 288 pixels).

Remarks This block extracts a rectangular region from the input image and copies it at a specified position in the output image. For example, it can be employed to move a blob (e.g., a human silhouette) whose bounding rectangle has been computed. The rectangle specifying the region that has to be extracted is provided as a 2x2 matrix. The first row of such matrix contains the x and y coordinates of the top-left corner of the rectangle; the second row contains the x and y coordinates of the bottom-right corner. The matrix must be a matrix of integer numbers. The output image is a black image having the same dimensions and properties of the input image. The extracted


rectangle is copied onto it in the position specified through the block’s parameters. In particular, the rectangle is copied so that the point specified through the “Reference point” parameter (the rectangle center, top left, or bottom left point) is in the output image at the position (in pixels) specified by the two remaining parameters. In case the input matrix is invalid of the rectangle it specifies has invalid dimensions, the block does not perform the copy and a black image is returned as output. This block is based on the Intel Image Processing Library (IPL). Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of its inputs. The input rectangle must be provided as a 2x2 matrix.

The input image rectangle must be provided as a 2x2 matrix of integer numbers. The first row of such matrix contains the x and y coordinates of the top-left corner of the rectangle; the second row contains the x and y coordinates of the bottom-right corner. Check the dimensions of your matrix.

The input rectangle must be provided as a matrix of integer numbers.

The input image rectangle must be provided as a 2x2 matrix of integer numbers. Check the type of your input matrix. If it is a matrix of floating point numbers you can convert it using the blocks included in the EyesWeb Math Library.

During the execution phase Message Description An error occurred in the locking phase. The block could not lock one of its inputs or its output.

The execution phase will fail. An error occurred while getting an ipl image. The block could not obtain a pointer to the input or

output image. The execution phase will fail. An error occurred while allocating an ipl ROI. The block could not allocate a Region Of Interest (ROI)

corresponding to the selected rectangular region in the input or output image. The execution phase will fail.

Warning: The x coordinate parameter is out of range. The bounding rectangle will be pasted in the center of the image.

The provided x coordinate is invalid, i.e., it is negative or larger than the image width. The extracted rectangular region will be copied in the center of the output image.

Warning: The y coordinate parameter is out of range. The bounding rectangle will be pasted in the center of the image.

The provided y coordinate is invalid, i.e., it is negative or larger than the image height. The extracted rectangular region will be copied in the center of the output image.

In setting the parameters of the block Message Description Warning: Invalid value of the x coordinate parameter. The old value will be kept.

The provided x coordinate is invalid, i.e., it is negative or larger than the image width. The old value will be kept.

Warning: Invalid value of the y coordinate parameter. The old value will be kept.

The provided y coordinate is invalid, i.e., it is negative or larger than the image height. The old value will be kept.

2.8 MultiClickDisplay CLSID: 0xFF6F2340,0xA969,0x4950,{0xA2,0x17,0xE8,0xB7,0xC1,0x51,0x8A,0x47}


The MultiClickDisplay block

Input Name Datatype Description Input image Imaging.Image The input image. The user can select a given number of points

in the image by clicking on them and get their coordinates as output.

Output Name Datatype Description Output coordinates

MathDatatype.Matrix A 2 x N matrix containing in its rows the x and y coordinates of the N points selected by the user.

Parameters Name Type Design/Run

time Description

Number of points

Integer [1, +∞) Design time The number of points whose coordinates have to be retrieved by clicking on the input image. The default value is 11 (the number of body joints, excluded the center of gravity, commonly considered in motion analysis applications).

Skip frame Command Run time When a “Skip frame” command is invoked the current frame is skipped and the every item of the output matrix is set to –1. Notice that this parameter is enabled only at run time.

Not available Command Run time When a “Not available” command is invoked the current point in the current frame is considered as not available and the items in its row of the output matrix are set to –1. Notice that this parameter is enabled only at run time.

Reset Command Run time When a “Reset” command is invoked, all the points in the current frame selected up to the moment in which the command is invoked are reset. The user will have to restart selecting points. Notice that this parameter is enabled only at run time.

Undo Command Run time When an “Undo” command is invoked the last selected point is erased and the user will have to select it again. “Undo” can be invoked more times until all the selected points are erased. Notice that this parameter is enabled only at run time.

Done Command Run time When the expected number of points has been reached, the user has to invoke the “Done” command in order to obtain the coordinates of the points in the output matrix. Notice that the “Done” command is enabled only when all the points have been selected. If “Undo” or “Reset” is invoked before “Done” the last or all the points are erased and the user has to select them again. The “Done” command will be again disabled until the reselection process is completed.

Draw points Boolean Design time Run time

If this parameter is set to “True” the selected points are drawn on the image in the click display window. The default value is “True”.

Thickness Integer [1, +∞) Design time Run time

The thickness of the drawn points. The default value is 2. This parameter is enabled only if points


are drawn (i.e., the “Draw points” parameter is set to “True”).


When a “Set color” command is invoked a dialog box is opened allowing the user to select a color for drawing.

Minimize with GUI


As in the standard Display block, when this parameter is set to “True”, the display window is minimized with the EyesWeb application; the window remains visible otherwise. The default value is “True”.

Interpolation Combo Design time Run time

As in the standard Display block, this parameter indicates which interpolation algorithm to use when the window and image sizes differ: from the least-CPU-intensive to the most-CPU-intensive the following algorithms are available: “Nearest neighbor”, “Linear”, and “Cubic” (which gives the best results). The default value is “Nearest neighbor”.

Remarks This block allows the user to select a given number of points on the input image by clicking on them with the mouse and to get their x and y coordinates as output. The number of points is chosen through the “Number of points” parameter. The output is an N x 2 matrix of integer numbers where each row contains the x and y coordinates of one of the selected points. Commands are provided to help the user in selecting points: when “Skip frame” is invoked, the whole output matrix is filled with –1 (thus indicating invalid points); when “Not available” is invoked, the current row of the output matrix (corresponding to the current point) is set to –1 (thus indicating that that point is invalid); when “Reset” is invoked all the points selected so far are dropped and the selection process has to restart; when “Undo” is invoked the last selected point is dropped and it has to be selected again. Multiple undo is supported (until all the selected points are dropped). Once the user selected the expected number of points, he/she has to invoke the “Done” command in order to get their coordinates as output. The “Done” command is enabled only at the end of the selection process. At that point, if “Reset” or “Undo” are invoked before “Done”, the “Done” command will be again disabled and the selection process will have to be repeated totally (with “Reset”) or partially (with “Undo”). The selected point can be drawn on the input image (displayed in the window of the click display block). Color and thickness of the drawn points can be selected through the corresponding parameters. Notice that this is an active block: it is activated both when a new image arrives and when an output is generated after invoking the “Done” command. If new frames arrive when the user is still selecting points on an older frame the new frames are dropped. The block should be used together with a passive MultimediaFileReader generating a new frame each time the user finishes to process the previous one (see the Figure below). A possible use of this block consists in the manual identification of body joints in a movie, when it is too difficult to extract them automatically through computer vision techniques (e.g., due to difficulties in performing background subtraction or motion tracking).

Figure: a patch using the MultiClickDisplay block: the passive MultimediaFileReader generates a new frame only when the user finishes to select points in the previous one.


Error and warning messages During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of its inputs. Failed in creating wake-up event. The block could not create a Win32 event. Failed in creating a temporary image buffer. The block could not create a temporary image buffer. Failed in initializing a temporary image buffer. The block could not initialize a temporary image buffer. Failed in initializing the image window. The block could not initialize a window for displaying

the input image During the execution phase Message Description An error occurred while locking the input. The block could not lock its input.

The execution phase will fail. An error occurred while locking the output. The block could not lock its output.

The execution phase will fail. An error occurred while getting the input ipl image. The block could not obtain a pointer to the input image.

The execution phase will fail. In managing the block’s private data Message Description An error occurred while saving private data. The block could not correctly save its private data: the




3 Add-ons to the EyesWeb Math Library

3.1 CrossCorrelation CLSID: 0xA0E193E2,0x93FF,0x44A9,{0x99,0x39,0x14,0x8A,0x0B,0xD7,0xA5,0x78}

The CrossCorrelation block

Inputs Name Datatype Description In1 MathDatatype.

Matrix A vector containing samples of the first signal involved in the cross-correlation operation. It must be an nx1 or 1xn vector (not a matrix).

In2 MathDatatype.Matrix

A vector containing samples of the second signal involved in the cross-correlation operation. It must be an nx1 or 1xn vector (not a matrix).

Output Name Datatype Description Out MathDatatype.

Matrix A vector containing the estimated cross-correlation. It is a vector of floating-point numbers, having “High lag” – “Low lag” + 1 items. The first item (Out[0]) is the estimated cross-correlation at a lag of “Low lag” samples; the last element (Out[“High lag” – “Low lag”]) is the estimated cross-correlation at a lag of “High lag” samples. The output is a row vector if the “Row vector” parameter is set to “True”, otherwise it is a column vector.

Parameters

Name Type Design/Run time Description Low lag Integer (-∞, +∞) Design Time The bottom of the range of lags at which cross-

correlation estimates should be computed. The default value is 0. See the remarks section for more details on how this parameter is used in computing cross-correlation.

High lag Integer (-∞, +∞) Design Time The top of the range of lags at which cross-correlation estimates should be computed. The default value is 10. See the remarks section for more details on how this parameter is used in computing cross-correlation..

Row vector Boolean Design Time

If this parameter is set to “True” the output is a row vector (i.e., a vector having 1 x “High lag” – “Low lag” + 1 items); otherwise the output is column vector (i.e., a vector having “High lag” – “Low lag” + 1 x 1 items). Default value is “False”.

Remarks This block estimates the cross-correlation of two input signals whose samples are contained in its two input vectors. These must be n x 1 or 1 x n vectors (not matrices). If u and v are the input vectors having n and m items (samples) respectively, cross-correlation is estimated as follows:

lag Lowlag Highk vukRn

ilag Lowkiivu −≤≤=∑

−

=++ 0)(

1

0,

where

≤≤

=otherwise

mh vv h

h 00

It must be “High lag” – “Low lag” ≥ 0.


The output is a vector of floating-point numbers, having “High lag” – “Low lag” + 1 items. The first item (Out[0]) is the estimated cross-correlation at a lag of “Low lag” samples; the last element (Out[“High lag” – “Low lag”]) is the estimated cross-correlation at a lag of “High lag” samples. The output is a row vector if the “Row vector” parameter is set to “True”, otherwise it is a column vector. In case one (or both) of the input vectors is invalid the output is also set as invalid. This block is based on the Intel Signal Processing Library (SPL). Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. It must be High Lag >= Low Lag It must be “High lag” – “Low lag” ≥ 0. Check the values

you provide for the “High lag” and “Low lag” parameters. The inputs must be 1 x n or n x 1 vectors. The inputs must be n x 1 or 1 x n vectors (not matrices).

Check your input vectors. During the execution phase Message Description An error occurred in the locking phase. The block could not lock an input or an output. The


3.2 DotProduct CLSID: 0x81C4A154,0xA7E4,0x43DC,{0xBA,0xB0,0x7C,0x7D,0x30,0xEC,0x30,0xC6}

The DotProduct block

Inputs Name Datatype Description In1 MathDatatype.

Matrix One of the two vectors on which the dot product has to be computed. It must be an nx1 or 1xn vector (not a matrix).

In2 MathDatatype.Matrix

The other vector on which the dot product has to be computed. It must be an nx1 or 1xn vector (not a matrix). The two vectors must have the same dimensions.


Scalar The calculated dot product of the two input vectors. It is a floating-point scalar if at least one of the two vectors is a vector of floating-point numbers; otherwise it is an integer scalar.

Parameters This block does not have any parameter. Remarks This block calculates the dot product of the two input vectors. These must be n x 1 or 1 x n vectors (not matrices) and they must have the same dimensions. If u and v are the input vectors having n items, the output will be:

∑=

=n

kkkvuvus

1),(


The output is a floating-point scalar if at least one of the two vectors is a vector of floating-point numbers; otherwise if both vectors are vectors of integers, it is an integer scalar. If one or both inputs are invalid, the output is also set to invalid. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The two input vectors must have the same dimensions.

The input vectors have different dimensions. They must be n x 1 or 1 x n vectors (not matrices) and they must have the same dimensions. Check your input vectors.

You must provide nx1 or 1xn vectors as input, not matrices.

You provided a matrix as input instead of a vector. Your input must be two n x 1 or 1 x n vectors. Check your input vectors.



3.3 FFT CLSID: 0x01617920,0xF18E,0x4EA1,{0x90,0x03,0xC2,0x4D,0x6F,0xDD,0x53,0x0A}

The FFT block Input Name Datatype Description Input signal MathDatatype.

Scalar The (scalar) input signal the Fast Fourier Transform (FFT) of which should be computed.

Outputs Name Datatype Description FFT Frequencies

MathDatatype. Matrix

A vector containing the frequencies associated to each item of the FFT output. Being N the number of samples in the input buffer (according to the “Buffer Length” parameters), this vector contains N/2 + 1 items, if “Extended FFT” is set to “False”, or N items if it is set to “True”. Note that frequencies are expressed in relative terms in the interval [0, 0.5] (usual FFT) or [0, 1] (extended FFT), i.e., they are divided by the sampling frequency.

FFT real part or Magnitude


A vector containing the real component or the magnitude of the computed FFT (depending on the “Output coordinates” parameter). Being N the number of samples in the input buffer (according to the “Buffer Length” parameters), this vector contains N/2 + 1 items, if “Extended FFT” is set to “False”, or N items if it is set to “True”.

FFT imaginary part or phase


A vector containing the imaginary component or the phase of the computed FFT (depending on the “Output coordinates” parameter). Being N the number of samples in the input buffer (according to the “Buffer Length” parameters), this vector contains N/2 + 1 items, if “Extended FFT” is set to “False”, or N items if it is set to “True”.


Parameters

Name Type Design/Run time Description Buffer Length Combo Design Time The dimension (in samples) of the buffer for the

input signal on which the FFT will be calculated. Allowed dimensions are the following: 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096 samples. The default value is 128.

Output coordinates

Combo Design Time Run Time

If “Polar” is chosen the block will return as output two vectors containing the magnitude and the phase of the computed FFT. If “Cartesian” is selected the block will return as output two vectors containing the real and the imaginary components of the computed FFT. The default value is “Polar”.

Extended FFT Boolean Design Time If this parameter is set to “True” the FFT is extended to the whole range of frequencies from 0 to the sampling frequency; otherwise the usual range from 0 to half of the sampling frequency is considered.

Row vectors Boolean Design Time

If this parameter is set to “True” the output vectors are row vectors; otherwise they are column vectors. Default value is “False”.

Remarks This block computes the Fast Fourier Transform (FFT) on the input signal. It stores the input samples in a buffer whose dimension is set through the “Buffer Length” parameter. Due to the implementation of the FFT algorithm only dimensions that are power of 2 are allowed. In particular the following dimensions are available: 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096 samples. The buffer is managed as a circular queue, i.e., when a new sample arrives, the oldest one is dropped down. At the beginning of the execution of a patch, the block waits until the buffer is full before computing the FFT. This means that the outputs will remain invalid until enough samples are received to completely fill the buffer. Notice that if the sampling frequency is not very high and the buffer is quite big, this can take quite a long time. For example, with a sampling frequency of 25 Hz (standard sampling frequency for video in Europe) and a buffer of 128 samples, 128/25 = 5.12 seconds are needed to fill the buffer and get the first valid output. In case an invalid input is received, the output is also set as invalid and the buffer is flushed. The block will start to fill the buffer again as soon as new valid inputs are received. The same time duration elapsed at the starting of the patch is therefore needed to obtain again a valid output after new valid inputs are received. The block returns 3 output vectors. They contain N/2 + 1 items, if “Extended FFT” is set to “False” (i.e., the usual range of frequencies from 0 to half of the sampling frequency is considered for the FFT), or N items if it is set to “True” (i.e., the FFT is extended to the full range between 0 and the sampling frequency). The first of the output vectors contain the relative frequencies (i.e., divided by the sampling frequency and therefore in the range [0, 0.5] or [0, 1] according to the “Extended FFT parameter) associated to the items of the FFT vectors. The other 2 vectors contain the computed FFT of the input signal: in case a polar coordinate system is selected (through the “Output coordinates” parameter) these two vectors will contain the magnitude and the phase of the FFT; in case of Cartesian coordinate system they will contain the real and imaginary components of the FFT. The outputs can be row vectors or column vectors depending on the “Row Vector” parameter. This block is based on the Intel Signal Processing Library (SPL). Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. During the execution phase Message Description An error occurred in the locking phase. The block could not lock the input or an output. The



3.4 FIRFilter CLSID: 0x6DCBB9E6,0x8024,0x4DD5,{0x89,0x72,0x69,0xA0,0x90,0x92,0x33,0xDC}

The FIR filters Input Name Datatype Description Input MathDatatype.

Scalar The (scalar) input signal that has to be filtered.

Output Name Datatype Description Output MathDatatype.

Scalar The filtered signal.

Parameters

Name Type Design/Run time Description Cut-off frequency or Low frequency

Double (0, 0.5] Design Time Run Time

When a low-pass filter or a high-pass filter is selected (see the “Filter type” parameter) this is the cut-off frequency of the low-pass or high pass filter respectively. When a band-pass filter is selected this is the low bound in frequency of the filter band. Frequencies are expressed in relative terms, i.e., with respect to the sampling frequency. Thus the allowed range is between 0 and 0.5 (i.e., half of the sampling frequency). The default value is 0.1.

High frequency or Unused

Double (0, 0.5] Design Time Run Time

When a band-pass filter is selected (see the “Filter type” parameter) this is the high bound in frequency of the filter band. Otherwise, the parameter is disabled and its label is set to “Unused”. Frequencies are expressed in relative terms, i.e., with respect to the sampling frequency. Thus the allowed range is between 0 and 0.5 (i.e., half of the sampling frequency). The default value is 0.4.

Number of taps Integer [5, +∞) Design Time

The number of coefficients of the filter, i.e., the number of coefficients of the impulse response of the filter. The default value is 10.

Smoothing window type


The smoothing window (if any) that should be used to smooth the filter response. Available options are: “Optimal Blackman window”, “No smoothing”, “Bartlett window”, “Hamming window”, and ”Hann window”. The default value is “Optimal Blackman window”.

Filter Type Combo Design Time Run Time

The type of filter that should be used. Available options are: “Low-Pass”, “High-Pass”, and “Band-Pass”. The default value is “Low-Pass”.

Normalize taps values

Boolean Design Time Run Time

If this parameter is set to “True” the coefficients of the impulse response of the filter are normalized, i.e., the response is 1 near the zero frequency. The default value is “False”.

Remarks This block implements Finite Impulse Response (FIR) filters. The input signal is filtered by the selected filter and is returned as output. The block automatically creates a filter according to the current configuration of parameters. In particular, the filter type (low-pass, band-pass, high-pass) can be selected through the “Filter Type” parameter, the band of the filter can be chosen through the “Cut-off frequency” parameter (low-pass and


high-pass filters) or the “Low frequency” and “High frequency” parameters (band-pass filter). Frequencies are always expressed in relative terms, i.e., they are divided by the sampling frequency. Thus the range of allowed frequencies is between 0 (not included) and 0.5 (i.e., half of the sampling frequency). Depending on the selected number of coefficients (“Number of taps” parameter) the block computes an ideal impulse response of the selected filter and smoothes the response data by the window specified by the “Smoothing window type” parameter. When starting the execution of a patch, the block has to wait for delay line of the filter to be full before producing its output. In the meanwhile the output is set as invalid. If the current input is invalid, the output is set as invalid too. In this case the block will wait for the first valid input: when the first valid input is received, the block starts again filling the delay line of the filter. In the meanwhile, the output will remain invalid. The same happens if one of the run-time parameters is changed during the execution of the patch: the coefficients of a new filter are computed and the block has to fill the delay line of the new filter. Notice that depending on the selected number of taps and on the sampling frequency the time needed for filling the delay line of the filter could be quite long. For example, with a sampling frequency of 25 Hz (standard sampling frequency for video in Europe) and 10 taps, 0.4 seconds are needed. This block is based on the Intel Signal Processing Library (SPL). Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. During the execution phase Message Description An error occurred in the locking phase. The block could not lock the input or the output. The

execution phase will fail. While creating a filter or modifying its parameters Message Description It must be Low frequency < High frequency: the filter cannot be created.

You are trying to create a band-pass filter but the value of the “Low frequency” parameters is larger than the value of “High frequency” parameter. The filter will not be created: if you run the patch anyway, you will get an invalid output. Check the values for the low and high frequencies.

It must be Low frequency < High frequency: the old values will be maintained.

You are trying to modify the settings of a band-pass filter but the value of the “Low frequency” parameters is larger than the value of “High frequency” parameter. The old values will be kept. Check the values for the low and high frequencies.

3.5 Hysteresis CLSID: 0xE7ECD53B,0x646C,0x4CD6,{0x87,0x73,0x26,0x9D,0xBA,0x76,0x06,0x22}

The Hysteresis block Input Name Datatype Description In MathDatatype.

Scalar The current input. It can be either an integer or a floating-point scalar.



Scalar The output calculated according to the required hysteresis. It can be one of two possible values (“Output High” or “Output Low”). Such values are passed as parameters.

Parameters

Remarks

Name Type Design/Run time Description Upward Threshold

Double (-∞, +∞) Design Time Run Time

The current value of the upward threshold. When the input value crosses this threshold (in the upward direction) the output is set to “Output High”. The default value is 0.3.

Downward Threshold

Double (-∞, +∞) Design Time Run Time

The current value of the downward threshold. When the input value crosses this threshold (in the downward direction) the output is set to “Output Low”. The default value is 0.6.

Output Low Double (-∞, +∞) Design Time Run Time

The current value of “Output Low”. The default value is 0.

Output High Double (-∞, +∞) Design Time Run Time

The current value of “Output High”. The default value is 1.

This block implements a hysteresis cycle as the ones depicted in the Figure below.

Downward Threshold Upward Threshold

Output Low

Output High

Downward ThresholdUpward Threshold

Output Low

Output High

When the current input crosses the upwaWhen it crosses the downward thresholdcurrent input is invalid, the output is alsvalid input arrives, threshold crossing is execution the output is set to “Output Loif the first input is higher than the highesthe two thresholds. In this case, the outpuntil the initial ambiguity is solved. Error and warning messages During the init phase Message An error occurred while getting the init d During the execution phase Message An error occurred in the locking phase.

Figure: Hysteresis cycles

rd threshold in the upward direction, the output is set to “Output High”. in the downward direction, the output is set to “Output Low”. If the o set to invalid. The last valid input is kept as reference: when a new evaluated with respect to the last valid input before it. During the first

w” if the first input is lower than the lowest threshold, to “Output High” t threshold, or to invalid if the first input falls into the region in between ut will remain invalid until one of the two thresholds is crossed, i.e.,

Description ata. The block could not obtain the init data of its input.

Description The block could not lock its input or its output. The execution phase will fail.


3.6 PointInRect CLSID: 0x8E2F66E9,0x29BE,0x45FE,{0x86,0x48,0x17,0x1C,0x7F,0x7B,0x47,0xC8}

The PointInRect block

Inputs Name Datatype Description InPoint MathDatatype.

Matrix The input point. It must a 1x2 vector.

InRectangle MathDatatype.Matrix

The input rectangle. It must be an 2x2 matrix.


Scalar The output is set to 1 if the input point belongs (i.e., is inside) the input rectangle, 0 otherwise.

Parameter

Name Type Design/Run time Description Include borders Boolean Design Time

Run Time Points lying on the borders of the input rectangle are considered as belonging to it only if this parameter is set to “True”. The default value is “True”.

Remarks This block checks if the current input points belongs to the current input rectangle. If yes, it returns 1, otherwise it returns 0. The input point must be provided as a 1x2 vector (its x and y coordinates). The input rectangle must be provided as a 2x2 matrix, the first row being the x and y coordinates of the top-left vertex and the second row being the x and y coordinates of the bottom-right vertex. If one or both inputs are invalid, the output is also set as invalid. If “Include borders” is set to “True”, points lying on the borders of the rectangle are considered as belonging to it. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The input point should be provided as a 1x2 vector.

The input point must be provided as a 1x2 vector containing the x and y coordinates of the point. Check the dimensions of your input points.

The input rectangle should be provided as a 2x2 matrix.

The input rectangle must be provided as a 2x2 matrix, the first row being the x and y coordinates of the top-left vertex and the second row the x and y coordinates of the bottom-right vertex. Check the dimensions of your input rectangles.




3.7 ProgressiveVector CLSID Active Version: 0x5CDF6B5B,0x124F,0x4C27,{0x82,0xDD,0xE2,0xA2,0x9B,0x3D,0xB7,0x18} CLSID Passive Version: 0x4164E12A,0xD2D4,0x411D,{0xA8,0xB8,0xAA,0x87,0xE4,0x19,0x27,0xBC}

The ProgressiveVector blocks

Input Name Datatype Description Sync in Any Only in the passive version: the synchronization input. The block is

activated each time something arrives on this input. Output Name Datatype Description Vector out MathDatatype.

Matrix The generated output vector.

Parameters

Name Type Design/Run time Description Period Integer

[1, +∞) Design Time Run Time

Only in the active version: the period of activation of the block in milliseconds. The block is activated every time the indicated period elapses. The default value is 100 ms.

Dimension Integer [1, +∞)

Design Time The number of items of the output vector. The default value is 10.

Starting number

Integer (-∞, +∞)

Design Time Run Time

The value of the first item of the output vector. The remaining items are progressive integer numbers, starting from this one. The default value is 0.

Row vector Boolean Design Time If set to “True”, the output vector is a row vector, i.e., a vector of 1 x “”Dimension” items. Otherwise, it is a column vector of “Dimension” x 1 items. The default value is “False”.

Remarks This block generates a vector of progressive integer numbers in the range [“Starting number”, “Starting number” + “Dimension” – 1]. For example, if “Starting number” = 0 and “Dimension” = 10, the block will generate the following vector [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]. In case the active version is used a new vector is generated each time the specified “Period” elapses. If the passive version is used, a new vector is generated each time a new input arrives in the synchronization input. Depending on the “Row vector” parameter a 1 x “Dimension” vector (“Row vector” = “True”) or a “Dimension” x 1 vector (“Row vector” = “False”) is generated. Error and warning messages During the execution phase Message Description An error occurred while locking the output. The block could not lock its output. The execution phase

will fail.


4 Support to FreeFrame plugins EyesWeb 3.3.0 includes support for FreeFrame plugins. FreeFrame (www.freeframe.org) is an open-source cross-platform real-time video effects plugin system. It provides a framework for developing video effects plugins and hosts on Windows, Linux and Mac OSX. The EywFreeFrame module allows users to load such plugins in EyesWeb. Plugins are loaded as EyesWeb blocks (see figure below) and appear in the EyesWeb tree-view in the Imaging.FreeFramePlugins folder. A sample plugin (EywFFSamplePlugin) is distributed with EyesWeb 3.3.0. Further plugins can be downloaded from the FreeFrame website.

FreeFrame plugins as EyesWeb blocks

4.1 Registering FreeFrame plugins The EywFreeFrame module is a generic host. It needs to be made aware of the FreeFrame plugins it must expose to EyesWeb. So it expects them to be placed in a FreeFramePlugins directory located in the same directory where the binary EywFreeFrame.dll is placed. However, this does not prevent the user to keep the plugins in the folder he/she prefers on his/her machine: the host looks also for plugins placed in the folder whose path is recorded in a FreeFrame key in the Windows registry (in HKEY_LOCAL_MACHINE\Software). In case FreeFrame plugins are placed in a FreeFramePlugins directory located in the same directory where the binary EywFreeFrame.dll is placed, you do not need to perform any further step. In case FreeFrame plugins are placed in another user-selected folder, a software tool is provided for helping users in creating the needed registry key. For registering the path of the folder containing FreeFrame plugins:

- Execute RegisterFreeFramePlugin.exe in the Dev\Tools folder of the EyesWeb distribution (the default location is C:\Program Files\EyesWeb 3.3.0)

- Select “Register FreeFrame Plugins” in the dialog box - Select the folder in which FreeFrame plugins are contained - Press “Register” - You will get a message saying whether the registration succeeded or not.

Note that on Windows 2000 and XP administrator privileges are usually needed to successfully perform the previous steps. The following figure shows the user interface of the registration tool.

The user interface of the FreeFrame Plugins registration tool. For registering the path of the plugins folder, select the folder browsing the folder list and press “Register”.

http://www.freeframe.org/


Note that it is possible to have plugins in both the FreeFramePlugins directory located in the same directory where the binary EywFreeFrame.dll is placed and in a user-selected directory registered as explained above. However, the dll files containing the plugins in the two folders must have different name. If two dlls have the same name, only the one contained in the FreeFramePlugins directory will be loaded into EyesWeb. Plugins can also be contained in subfolders of the FreeFramePlugins folder and/or of the user-selected folder. Plugins in subfolders will appear in the EyesWeb tree-view in Imaging.FreeFramePlugin.SubfolderName. The EyesWeb 3.3.0 installation program automatically registers the EywFreeFrame.dll file during the installation. Each time the EywFreeFrame.dll file is registered, a certain number of EyesWeb modules are generated, corresponding to the number of FreeFrame plugins found in the system (in the above mentioned folders). This means that if new plugins are added to the system, the dll must be registered again to make EyesWeb aware of the new plugins. For registering the EywFreeFrame.dll file, simply execute the batch file RefreshFreeFramePlugins.bat that is located in the same directory where the EyesWeb executable file is (the default location is C:\Program Files\EyesWeb 3.3.0\Binaries). Note that EyesWeb must be quit and restarted again to see the changes in the number of available FreeFrame plugins (or ReloadLibraries must be chosen from the System menu, when no patch is opened).

4.2 Retrieving FreeFrame plugin information Information about the FreeFrame plugin wrapped by a given EyesWeb block and the FreeFrame API the plugin emplyes can be obtained from the about dialog of the EyesWeb block. In particular, two buttons are available:

- The “FF Version” button opens a dialog box reporting information about the FreeFrame API version the plugin employs (see figure below).

- The “Plugin info” button opens a dialog box reporting information about properties of the specific FreeFrame plugin such as plugin name and type, supported video formats, plugin version number etc. (see figure below).

Dialog windows reporting information about a plugin and the version of the FreeFrame API the plugin employs.

4.3 Inputs, output, and parameters FreeFrame plugins are loaded as EyesWeb blocks in the Imaging.FreeFrame folder and its subfolders. User can use them as standard EyesWeb blocks in their patches. Two kinds of FreeFrame plugins exist: sources, i.e., plugins generating new images (e.g., frame grabbers) and effects, i.e., plugins processing incoming images (e.g., video effects). Depending on the kind of plugin, the corresponding EyesWeb block has different inputs and parameters. Inputs EyesWeb blocks representing FreeFrame plugins usually receive images as inputs. That is, the input datatype is the Imaging.Image datatype. The format of the input images must be the following: 3 channels BGR or 4 channels BGRA, 8 bits per channel, pixel ordered (i.e., BGRBGRBGR…) images. Note that the host does not currently support images padded with zeros for dword or quadword alignment. Note also that even if the FreeFrame specification includes the RGB16 format, EyesWeb does not support such format. Therefore, plugins working only with the RGB16 format can be loaded but not executed in EyesWeb. If the user tries to execute a patch containing one of such plugins, an error message will be displayed during the init phase and it will be


impossible to lock the patch. If a plugin has more than one image input, all the input images must have the same format (i.e., same width, height, orientation, etc.). Blocks representing source plugins can have zero, one, or more inputs.

- No input: these are active blocks activated either periodically or on event. In case the block is periodically activated, the period can be set as parameter. If the block is activated on event, a “Get image” command parameter is available for activating the block. More information can be found in the parameters section.

- One input: if the block is a passive block, the input is the synchronization input. Even if this input can receive any datatype, it should be connected to the output of a clock block providing periodic bangs that activate the block. If the block is an active block, the input is an auxiliary image input. The format of such input image must be the one described above. If the input is a synchronization input it is named “Clock”. If it is an image, the name is “Input image 0”.

- More inputs: the block can be either active or passive with a synchronization input. All the inputs other than the (possible) synchronization input are auxiliary image inputs. The format of such input images must be the one described above. All the input images must have the same format. The (possible) synchronization input is named “Clock”. The image inputs are named with zero-based progressive numbers “Input image #”.

Block representing effect plugins are always passive blocks having one or more inputs. Inputs are always images (Imaging.Image datatype). The format of the input images must be the one described above. In case of more than one input, all the input images must have the same format. The image inputs are named with zero-based progressive numbers “Input image #”. Output FreeFrame block always have just one output. The output is always an image (Imaging.Image datatype) and it is named “Output image”. The format of the output image depends on the kind of plugin the block represents:

- If the block represents a source plugin without any auxiliary image input (i.e., an active block without input or a passive block with only the synchronization input), a “Source params” command parameter is available. When such command is invoked, the block opens a dialog window allowing the user to set some properties of the output image: width, height, format (RGB24 or RGB32), and orientation (top left or bottom left). The dialog window is shown in the figure below.

- If the block represents a source plugin with one or more auxiliary image input, the format of the output image is the same than the format of the input image(s).

- If the block represents an effect plugin, the format of the output image is the same than the format of the input image(s). If the plugin has only one input and it is optimized for in-place processing, the output is in-place.

The Source Parameters window Parameters Two groups of parameters are involved in EyesWeb blocks representing FreeFrame plugins: (i) parameters that directly correspond to parameters of the wrapped FreeFrame plugin and parameters of the hosting EyesWeb block. The former depend on the specific plugin the block wraps, the latter depend on the kind of plugin, i.e., whether it is a source or an effect plugin. Parameters in FreeFrame are usually floating-point numbers (float data type in C++) in the range [0, 1]. The current FreeFrame specification defines 9 parameter types all using the same float data type except for strings. The EyesWeb host handles these parameter types in the following way:

- FreeFrame Standard parameters are represented as EyesWeb Double parameters in the range [0, 1].


- FreeFrame Boolean parameters are represented as EyesWeb Boolean parameters. - FreeFrame Red, Green, Blu, Xpos, and Ypos parameters are represented as EyesWeb Double

parameters in the range [0, 1], where 0 corresponds to the lower limit of the range of the parameter and 1 to the upper limit. For example, in case of color parameters the range [0, 255] is mapped in [0, 1]; in case of position parameter the ranges [0, image width] and [0, image height] are mapped in [0, 1].

- FreeFrame Event parameters are represented as EyesWeb Command parameters. - FreeFrame Text parameters are not supported by the current version of the EyesWeb host.

The parameter names and the default values are those provided by the underlying FreeFrame plugin. In addition to FreeFrame plugin parameters, further parameters are available depending on the kind of loaded plugin (source or effect). Effect plugins have the following additional parameter:

Name Type Design/Run time Description Disable effect Boolean Design Time If “True” the effect plugin is disabled and the input

image coming into the first input of the block is returned as output. The default value is “False”.

Source plugins have the following additional parameters:

Name Type Design/Run time Description Sync in Boolean Design Time If “True” the source block will be a passive block with

synchronization input; otherwise it will be an active block activated on periodic basis or on event. The default value is “False”.

Activation mode

Combo Design Time In case of active block, this parameter allows to select the block’s activation mode. Two options are available: “Periodic” – the block is activated on periodic basis, and “On event” – the block is activated on event. Note that this parameter is enabled only if the “Sync in” parameter is set to “False”. The default value is “Periodic”.

Period Integer [1, +∞) Design Time Run Time

The period of activation of the block in milliseconds. The block is activated every time the indicated period elapses. Note that this parameter is enabled only if the “Sync in” parameter is set to “False” and the “Activation Mode” parameter is set to “Periodic”. The default value is 40 ms, corresponding to the usual European sampling rate of 25 fps.

Get image Command Design Time Run Time

The activation event of the block. Each time this command is invoked a new image is generated by the source block. Note that this parameter is enabled only if the “Sync in” parameter is set to “False” and the “Activation Mode” parameter is set to “On event”.

Source params Command Design Time Each time this command is invoked the Source Parameters dialog window is opened to allow the user to set some properties of the output image (see figure in the previous page). Note that this parameter is available only if the source block does not have any auxiliary image input.

4.4 Error and warning messages While instantiating the EyesWeb block Message Description The FreeFrame plugin has been incorrectly registered

The block could not find the path of the dll containing the FreeFrame plugin. A dummy block will be created.

Error in initializing the FreeFrame plugin The block could not correctly initialize the FreeFrame


plugin it wraps. The plugin could be buggy. Error in getting information about the FreeFrame plugin

The block could not get a description of the FreeFrame plugin it wraps. No description will be provided.

Error in loading the FreeFrame plugin: number of parameters unknown

The block could not get the number of parameters of the FreeFrame plugin it wraps. No parameter will be available.

Error in loading the FreeFrame plugin: unknown name of parameter

The block could not get the name of one of the parameters of the FreeFrame plugin it wraps. The parameter will labeled as “Unknown”.

Error in getting the default value of a parameter. It will be set to 0.

The block could not get the default value of one of the parameters of the FreeFrame plugin it wraps. The default value will be set to 0.

The FreeFrame plugin is not correctly initialized The block could not correctly initialize the FreeFrame plugin it wraps. The plugin could be buggy.

This plugin does not support RGB24 and RGB32 image formats: it cannot be used with EyesWeb!

The plugin supports only the RGB16 format, but EyesWeb does not support this format. If the plugin is used in a patch, the patch cannot be locked.

Note that all the errors generated while instantiating an EyesWeb block are not displayed in the EyesWeb message bar. Instead, they are communicated to the user through error message boxes. During the init phase Message Description An error occurred while getting the input init data. The block could not obtain the init data of one of its

inputs. This plugin does not support RGB24 and RGB32 image formats: it cannot be used with EyesWeb!

The plugin supports only the RGB16 format, but EyesWeb does not support this format: the plugin cannot be used with EyesWeb and the patch cannot be locked.

This plugin is an effect but does not have any input: it cannot be used with EyesWeb!

Effect plugins are implemented as EyesWeb passive block: therefore they must have at least one input. This message is likely to be generated by a buggy plugin.

The input images must be pixel ordered (RGBRGBRGB...)

The input image is required to be pixel ordered. You can use a conversion block in the Imaging.Conversion library to obtain a pixel ordered image.

Only the BGR color model is supported. The input image is required to be a BGR image. You can use a conversion block in the Imaging.Conversion library to obtain a pixel ordered image.

The input images are RGB32 images, but this FreeFrame plugin does not support this format.

The input image is a RGB32 image (i.e., 4 channels, 8-bit per channel image), but this format is not supported by the current FreeFrame plugin. You can use blocks in the Imaging.Operations library to convert the input image in a RGB24 image (i.e., 3 channels, 8-bit per channel image).

The input images are RGB24 images, but this FreeFrame plugin does not support this format.

The input image is a RGB24 image (i.e., 3 channels, 8-bit per channel image), but this format is not supported by the current FreeFrame plugin. Currently, there is not an easy way to convert an RGB24 image in a RGB32 image in EyesWeb. Try to convert the images before feeding them into EyesWeb.

Only RGB24 and RGB32 image formats are supported.

The format of the input image is not RGB24 or RGB32. Check the format of your input images and eventually convert them using the blocks contained in the EyesWeb Imaging library.

Only 8-bit unsigned images are supported. The input images are required to have a depth of 8 bits (unsigned). You can use conversion blocks in the Imaging.Operations library to convert the input images.

All the input images must have the same format. The block has more than one input, but the input images differ in their format. Check the format of your input images: all of them must have the same format. Eventually convert some of the input images using the


blocks contained in the EyesWeb Imaging library. Error in instantiating FreeFrame plugin. The block could not correctly instantiate the FreeFrame

plugin it wraps. The patch cannot be locked. FreeFrame plugin is not properly loaded. An error occurred while loading the dll containing the

FreeFrame plugin. The patch cannot be locked. Error in setting plugin parameters. The block could not initialize the FreeFrame plugin it

wraps with the current values of the parameters. The patch cannot be locked.


execution phase will fail. An error occurred while getting an ipl image. The block could not obtain a pointer to one of its input

or output images. The execution phase will fail. Aligned images padded with zeros are not supported. The host does not currently support images padded with

zeros for dword or quadword alignment. The execution phase will fail.

FreeFrame error in processing the input image. An error occurred while the FreeFrame plugin was processing the input image(s). The execution phase will fail.

While unlocking the patch Message Description Error in deinstantiating FreeFrame plugin. The block could not correctly deistantiate the FreeFrame

plugin it wraps. The patch cannot be correctly unlocked. In setting the parameters of the block Message Description Error in setting plugin parameters. The block could not set the FreeFrame plugin it wraps

with the value of the parameter just changed by the user. In managing the host’s private data Message Description An error occurred while saving private data. The host could not correctly save its private data: in case

of source plugins without auxiliary image inputs the information about the user’s selected properties of the output image will be lost.

An error occurred while loading private data. The host could not correctly load its private data: in case of source plugins without auxiliary image inputs, the default values for the properties of the output image will be used.

4.5 A sample plugin A sample plugin (EywFFSamplePlugin) is distributed with EyesWeb 3.3.0. This is an effect plugin having just one input and one parameter. It simply adds the value passed as parameter (and rescaled from [0, 1] to [0, 255]) to the R, G, and B values of each pixel in the input image. In fact, the sample plugin performs the same operation and can be compared with the EyesWeb Imaging Monadic Arithmetic Op block when “add” is chosen as operation type. The simplest way to use it is to connect its input with a MultimediaFileRead block and its output with an Imaging Display block: an example can be found in FreeFrameSample patch distributed with EyesWeb 3.3.0. Note that the MultimediaFileRead block usually generates plane ordered images. A conversion block is thus needed between the MultimediaFileRead block and the FreeFrame plugin block in order to obtain pixel ordered images.


5 Sample patches Examples of use of many of the blocks described in this document can be found in the sample patches coming with the EyesWeb 3.3.0 distribution. Here follows a brief summary of such sample patches with reference to the blocks they employ. Centered Silhouette Used block: MoveBoundingRect Description: this patch extracts the body silhouette and copies it on a new image in such a way that the silhouette is always at the center of the image and the lower border of the bounding rectangle coincides with the lower border of the image. Cross-correlation Used blocks: Counter, CrossCorrelation, ProgressiveVector Description: this patch shows the use of the cross-correlation block. Two gaussian shapes are sent as input to the cross-correlation block: a gaussian centered in 0.5 and a gaussian moved on the left or on the right as selected by the user through a slider. The output of the cross-correlation block will show a peak in correspondence of the lag that should be introduced in the template gaussian in order to match the moving gaussian. Draw Shapes Used blocks: DrawEllipse, DrawPolylines, DrawRectangles Description: this patch shows the use of some blocks for drawing shapes on images. In particular the patch includes blocks for drawing ellipses, rectangles, and polygons. FFT and Filters Used blocks: FIRFilter, FFT Description: this patch shows the use of FFT and filters on scalar signals. The input signal is a sinusoid whose frequency can be varied in the range [0, 5] Hz (being 10 Hz the sampling frequency). Such input signal and its spectrum are displayed as output. The input signal is then sent to a low-pass filter, a band-pass filter, and a high-pass filter. The signals returned by these filters are displayed together with their spectra. FreeFrame Sample Used block: sample FreeFrame plugin (EywFFSamplePlugin) Description: this patch shows an example of use in EyesWeb of a FreeFrame video plugin. The employed plugin, distributed with EyesWeb, adds a scalar value passed as parameter to the R, G, and B values of each pixel of the input image. Histogram Used blocks: ComputeHistogram, ProgressiveVector Description: this patch shows an example of use of the ComputeHistogram block. The histogram of the b/w silhouette of the dancer in the input movie is computed and displayed as output. The indexes on the x axis of the output histogram are obtained through the ProgressiveVector block. Points selection Used block: MultiClickDisplay Description: this patch allows the user to watch a movie frame by frame and to select a given number of points for each frame. The coordinates of the selected points are stored in a matrix returned as output. The selected points can then be saved in a text file by using the appropriate EyesWeb block (Math.Matrix.Output.LogToFile).

EyesWeb Motion Analysis Reference 1

Motion Analysis Library Reference This library is part of the EyesWeb software environment and can only be used in accordance with the Licence provided.

Copyright Notice The EyesWeb Motion Analysis Library is Copyright © 2001-2002, Laboratorio di Informatica Musicale - DIST - University of Genoa (www.infomus.dist.unige.it)

License agreement Use of the EyesWeb Motion Analysis Library (hereinafter 'SOFTWARE') is contingent on your agreement to the following terms: WARRANTY & USE: DIST - University of Genoa grants you a limited, non-exclusive license to use the SOFTWARE free of charge only for educational purposes and research at universities and government research laboratories. Companies and private research laboratories are required to obtain a separate license. For any other (not educational) use, commercial or private, it is also required to obtain a license. DIST - University of Genoa makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. DIST - University of Genoa is not obligated to provide maintenance or updates for the SOFTWARE. DISTRIBUTION: universities and government research laboratories may freely distribute the SOFTWARE, in original version, only to other universities and government research laboratories provided that this copyright and permission notice appear on all copies and supporting documentation, and that the DIST - University of Genoa copyright notices are referred in the following ways: a) DIST - University of Genoa's copyright notice should be included in the documentation, regardless of the media used to supply the documentation; b) The DIST - University of Genoa and EyesWeb Logos have to appear on packages and promotional material (DIST - University of Genoa makes the logos available on the EyesWeb ftp site ftp://infomus.dist.unige.it); c) in the 'about box' of the product, in the case that it is not the EyesWeb about box, DIST - University of Genoa and EyesWeb must be cited in the following manner: “EyesWeb is copyright (c) Laboratorio di Infomatica Musicale - DIST - University of Genoa (www.infomus.dist.unige.it )”, and the EyesWeb Motion Analysis Library must be cited in the following manner: “The EyesWeb Motion Analysis Library is copyright (c) Laboratorio di Infomatica Musicale - DIST – University of Genoa (www.infomus.dist.unige.it)”; (d) any public use of EyesWeb, of the EyesWeb Motion Analysis Library, or of any application based on EyesWeb or on the EyesWeb Motion Analysis Library, must be preliminarily notified to [email protected]. Distribution to companies, private research laboratories, or any other kind of subject must be preliminarily agreed with DIST. Antonio Camurri DIST - University of Genoa Viale Causa, 13 16145 Genova, Italy E-mail: [email protected] This work is based in part on the Intel IPL and OpenCV libraries, Copyright © Intel Corporation.

http://www.infomus.dist.unige.it/

ftp://infomus.dist.unige.it/






Table of Contents Motion Analysis Library ........................................................................................................................................ 1 Copyright Notice .................................................................................................................................................... 1 License agreement .................................................................................................................................................. 1 1 Cues ............................................................................................................................................................ 4

1.1 ContractionIndex .................................................................................................................................... 4 1.2 DirectnessIndex ...................................................................................................................................... 5 1.3 SMI (Silhouette Motion Image)............................................................................................................. 6 1.4 StabilityIndex.......................................................................................................................................... 8

2 Motion tracking ........................................................................................................................................ 10 2.1 CentroidsCalc ....................................................................................................................................... 10 2.2 CentroidsDraw...................................................................................................................................... 12 2.3 CentroidsExtract ................................................................................................................................... 14 2.4 CentroidsShow...................................................................................................................................... 15 2.5 LKTracker ............................................................................................................................................ 16 2.6 MatrixLKTracker.................................................................................................................................. 18 2.7 TrackerViewer ...................................................................................................................................... 19

3 Posture Recognition.................................................................................................................................. 22 3.1 DeformingNormalizer........................................................................................................................... 22 3.2 HuMoments .......................................................................................................................................... 23 3.3 HuPostureRecognizer ........................................................................................................................... 24 3.4 NotDeformingNormalizer..................................................................................................................... 26 3.5 SPMPostureRecognizer ........................................................................................................................ 28

4 SpaceAnalysis........................................................................................................................................... 30 4.1 HitCellDetector..................................................................................................................................... 30 4.2 ImageHitCellDetector........................................................................................................................... 31 4.3 PositionDependingPotentials ................................................................................................................ 32 4.4 RegionDependingPotentials.................................................................................................................. 35 4.5 SpaceAnalysisViewer ........................................................................................................................... 38

5 References ................................................................................................................................................ 40


EyesWeb Motion Analysis Library Version 2.0.0 – Released on May 7th, 2002. With the partial support of the IST Project 20410 MEGA (Multisensory Expressive Gesture Applications). With the partial support of the IST Project 32729 CARE HERE (Creating Aesthetically Resonant Environments for the Handicapped, Elderly and Rehabilitation). The Motion Analysis Library is contained in the EywMotionAnalysis.dll file. Some further extensions (blob color tracking, extensions to the EyesWeb imaging library) are contained in other DLL files and documented separately. Some blocks (HitCellDetector, PositionDependingPotentials, RegionDependingPotentials, SpaceAnalysisViewer) are also registered under the “Mapping” library, since they can be used to implement multimodal mapping strategies. The Motion Analysis Library is based in part on the Intel IPL and OpenCV Libraries. The Motion Analysis Library actually contains blocks and patches devoted to: (i) Obtain measures of low-level physical parameters (i.e., tracking of features on the body silhouette); (ii) Individuate and extract expressive cues: such expressive cues include global measures (i.e., cues

depending on full body movement) such as the contraction index and the quantity of motion, cues inspired by the Rudolf Laban’s Effort Theory (Laban, 1947, 1963) such as the directness index, cues inspired by psychological studies such as the durations of pause and motion phases (Wallbott 1980, Argyle, 1980);

(iii) Segment movement into gestures in order to study and recognize the expressive intentions associated with them. In this perspective, algorithms for recognition and segmentation of pause and motion phases are included as patches. Simple algorithms for recognition of body postures are included as well.


1 Cues

1.1 ContractionIndex CLSID: 0xAEDE4322,0x9A26,0x4A30,{0x81,0x8B,0xCA,0x62,0x33,0x6C,0x61,0x5F}

Inputs Name Datatype Input BW Image Imaging.Ima

Outputs Name Datatype Contraction Index MathDatatyp

Scalar Area MathDatatyp

Scalar Bounding Box MathDatatyp

Matrix

Parameters This block does not have any par Remarks The ContractionIndex block calcIn particular, it can calculate the cconsidered as a measure of how mshould receive as input an imagecalculated as the ratio between tobject’s bounding box. The block Error and warning messages During the init phase Message An error occurred while getting thJust single channel images allowe

Just 8 bit depth images allowed.

Figure 1.1: The ContractionIndex block

Description ge The input image on which the index has to be calculated. It must be a

B&W image containing the silhouette of the analyzed object.

Description e. The calculated contraction index. It is an integer scalar.

e. The calculated area (i.e., the number of pixels) of the analyzed object’s silhouette. It is an integer scalar.

e. The calculated bounding box of the analyzed object’s silhouette. It is a 2x2 matrix having the (x, y) coordinates of the top left point in its first row, and the coordinates of the bottom right point in its second row.

ameter.

ulates and returns as output the contraction index of a blob in the input image. ontraction index of a body silhouette. In this case, the contraction index can be uch a body posture is “open”. Notice that in order to work properly the block

containing a clean silhouette of the analyzed object. The contraction index is he area (i.e., the number of pixels) of the object’s silhouette and the area of returns also the area and bounding box of the object’s silhouette.

Description e init data. The block could not obtain the init data of its input. d. The input image must be single channel. Check the

properties of your input image. You can use a conversion block from the Imaging library to obtain a single channel image. The input image must have an 8-bit depth. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a 8 bit depth image.



execution phase will fail. An error occurred while getting the input ipl image.

The block could not obtain a pointer to the input ipl image. The execution phase will fail.

1.2 DirectnessIndex CLSID: 0x8E7749AA,0xD19C,0x4BD4,{0xAF,0x92,0x62,0x48,0x17,0x6D,0x96,0xB1}

k Inputs Name Datatype MoCap Input MoCap.

MoCapDataL

Outputs Name Datatype Directness Index MoCap.

MoCapDataL

Parameters Name Type Mode Combo

Samples Integer [2, +∞)

Reset

Command

Start/Stop Command

Figure 1.2: The DirectnessIndex bloc

Description

ist A list of MoCap data containing the incoming (x, y) coordinates of a collection of tracked points. The tracked points should be included as MoCap 2D points in the input MoCap list. If the input MoCap list contains also different kinds of items, only 2D points will be considered.

Description

ist A MoCap list containing the calculated directness indexes for each input point. The indexes are included in the output list as MoCap scalars. They are floating point scalars, having values in [0, 1]. Reliability is always set to 1. See the remark section for more information.

Design/Run time Description Design Time Run Time

A combo allowing the user to select the operating mode of the block. Two options are currently available: “Running”, i.e., the directness index is calculated each time a new input arrives, and “On command”, i.e., the directness index is calculated only when a “Start/Stop” command is invoked. The default value is “Running”.


The number of samples (points), with respect to which the directness index is calculated. This parameter is enabled only if the “Running” mode was selected. The default value is 10.


When a Reset command is invoked the points composing the current trajectories with respect to which the directness index is calculated are discarded and the block starts to store new trajectories. This parameter is displayed only if the “Running” mode was selected.


When a Start/Stop command is invoked the directness index is calculated on the trajectories so far stored. Then the current trajectories are discarded and the block starts to store new trajectories. This


parameter is displayed only if the “On command” mode was selected.

Remarks The DirectnessIndex block calculates the directness index on trajectories of (2D) tracked points. It receives as input a MoCap list containing the last position of each tracked point. The tracked points should be included as MoCap 2D points in the input MoCap list. If the input MoCap list contains also different kinds of items, only 2D points will be considered. Points having reliability equal to 0 are not considered and the corresponding trajectories are reset. Note that if the input MoCap list is not valid, the output will be also set as invalid. Each time a new input arrives the new coordinates of the tracked points are stored. A trajectory is composed by a temporal sequence of segments connecting the newly arrived coordinates with the previous ones. In such a way, a trajectory is stored for each tracked point. If the “Running” mode of operation was selected, trajectories up to the last N inputs are stores, where N is the value of the “Samples” parameter. Otherwise, if the “On command” mode of operation was selected, trajectories up to the last Start/Stop command are stored. The directness index is calculated for each stored trajectory as the ratio between the length of the straight trajectory connecting the first and last point of the stored trajectory and the sum of the lengths of each segment constituting the stored trajectory. Therefore, the more it is near to one, the more direct is the stored trajectory (i.e., the stored trajectory is “near” to the straight one). If the “Running” mode of operation was selected, the directness index is calculated each time a new input arrives on trajectories composed by segments connecting the last “Samples” positions of any tracked point. Otherwise, if the “On command” mode of operation was selected, the directness index is calculated each time a Start/Stop command arrives on the trajectories so far stored since the last Start/Stop command was invoked. Error and warning messages During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output. The

execution phase will fail. Warning: An error occurred while updating the stored trajectories.

An error occurred while adding new incoming points to the currently stored trajectories. The execution phase will not be stopped, but the output will be set as invalid. If possible, unlock the patch and restart it.

1.3 SMI (Silhouette Motion Image) CLSID: 0x6A3AB50B,0x22F6,0x4D6F,{0x85,0x86,0x12,0x50,0x1F,0xB3,0xE1,0xAB}

Figure 1.3: The SMI block Inputs Name Datatype Description Input Image Imaging.Image The current input image used to calculate the Silhouette Motion Image

and the quantity of motion. It must be a single channel image. It should contain an object’s silhouette. Usually, it contains a human silhouette.

Outputs Name Datatype Description SMI Imaging.Image The calculated Silhouette Motion Image. Quantity of Motion

MathDatatype. Scalar

A measure of the actual quantity of motion calculated using the Silhouette Motion Images. It is a floating point scalar.


Parameters Name Type Design/Run time Description Images Integer [2, +∞) Design Time The number of frames the block uses to calculate the

current Silhouette Motion Image. The last (“Images” – 1) frames and the current one are stored in an internal data structure. Increasing numbers for the “Images” parameter require an increasing amount of memory on your system.

Median Filter Integer [3, +∞) Design Time

The parameters for a median filter applied to the calculated Silhouette Motion Image in order to reduce noise. The same value is applied both to the number of rows and to the number of columns of the matrix used in the median filter. For more information see the Intel Image Processing Library (IPL) version 2.5 documentation.

Remarks The SMI block calculates a Silhouette Motion Image (Camurri, Trocca, Volpe, 2002). It returns also a measure of the actually detected quantity of motion. A Silhouette Motion Image is an image carrying information about variations of the silhouette shape and position in the last few frames. SMIs can be seen as a special case of Motion Templates (see OpenCV Reference Manual), where information about time is implicit in the image and not explicitly recorded. The SMI is generated by the following formula:

][][][_ tsilhouetteitsilhouettetimagemotion

i

−−= ∑

The motion image at frame t is generated adding images of the silhouette in the previous N frames, where N is determined by the “Images” parameter, and then subtracting the silhouette at frame t. The resulting image contains just variations happened in the previous frames. The actual quantity of motion is calculated as

Movement=Area(SMI[t,n])/Area(Silhouette[t])

The result can be thought as a rough approximation of the quantity of motion, i.e., q=m * v, where m is the mass and v stands for velocity. Of course the area of a SMI is not q, but the behavior is similar: actually the shape of the graph is close to the shape of the graphs of velocity of a marker put on a limb. Scaling the SMI area by the area of the most recent silhouette allows to obtain measures almost independent from the camera’s distance and expressed in terms of fractions of the body area that moved. For example, it is possible to say that at instant t a movement corresponding to the 2.5% of the total area covered by the silhouette happened. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. Failed to create a temporary image buffer. The block could not create a temporary image buffer.

Check if enough memory is available on your system. Failed to initialize a temporary image buffer. The block could not initialize a temporary image buffer.

Check if enough memory is available on your system. Just single channel images allowed. The input image must be single channel. Check the

properties of your input image. You can use a conversion block from the Imaging library to obtain a single channel image.

During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output. The



An error occurred while getting an ipl image. The block could not obtain a pointer to the input or output ipl image. The execution phase will fail.

1.4 StabilityIndex CLSID: 0x6829AAB5,0xE181,0x4D78,{0x8E,0x41,0x51,0xB0,0xCD,0xBF,0x63,0x5F}

Inputs Name Datatype MoCap Input MoCap.

MoCapDataList

Outputs Name Datatype Stability Index MathDatatype.

Scalar Parameters Name Type Expected CoG Label

String

Expected Left Foot Label

String

Expected Right Foot Label

String

Use always two lowest

Boolean

Is input flipped ?

Boolean

Figure 1.4: The StabilityIndex block

Description

A list of MoCap data containing the incoming (x, y) coordinates of the body joints. In particular, the coordinates of Center of Gravity and feet are needed to calculate the Stability Index. If the coordinates of the feet are not available the block can search for the two lowest points in the input list. The coordinates of Center of Gravity and feet should be included as MoCap 2D points in the input MoCap list.

Description The calculated value of the stability index. It is a floating-point scalar.

Design/Run time Description Design Time Run Time

The expected label of the incoming MoCap item containing the actual coordinates of the Center of Gravity. The default value is “No label”. See the remark section for more information.


The expected label of the incoming MoCap item containing the actual coordinates of the left foot. The default value is “No label”. See the remark section for more information.


The expected label of the incoming MoCap item containing the actual coordinates of the right foot. The default value is “No label”. See the remark section for more information.


If set, the block instead to consider the coordinates of the feet, always searches for the two lowest points in the input MoCap list. This means that the two points having the maximum y coordinates are selected (if, as usual, the coordinates are referred to a top-left origin). The default value is False.

Design Time

It should be set to True if the incoming 2D points coordinates are referred to a bottom-left origin. In this case, if the blocks searches for the two lowest points, the points having the minimum y coordinates are selected. The default value is False.


Remarks The StabilityIndex block calculates the index of stability starting from the previously measured positions of body joints. It should receives as input a list of MoCap data containing the actual (x, y) coordinates of the body joints. In particular, the coordinates of Center of Gravity and feet are needed to calculate the Stability Index. They should be included as MoCap 2D points in the input MoCap list. If the coordinates of the feet are not available the block can search for the two lowest points in the input MoCap list. Note that if the input MoCap list is not valid, the output will be also set as invalid. Since precise and reliable measures of the positions of body joints are not currently available in EyesWeb, they can be replaced by the coordinates of the body centroids (MotionAnalysis.FeatureTrackers.BodyCentroids block). Notice that body centroids are a very rough approximation of body joints and therefore they could be unreliable. As a consequence, the stability index will be also unreliable when body centroids fail in approximating body joints. The block first searches the input MoCap list for the coordinates of the Center of Gravity. It tries to find them by using the expected label passed as parameter. If this search fails, the block tries again by using a set of predefined labels, namely “CoG”, “COG”, “CenterOfGravity”, “Center Of Gravity”. If again the Center of Gravity cannot be found, a warning message is displayed and the output is set as invalid. The same mechanism applies to the search for the feet coordinates. In this case, the predefined labels the block searches for are “Left Foot”, “LF”, “Foot Left”, “FL”, “Foot_Lx” for the left foot, and “Right Foot”, “RF”, “Foot Right”, “FR”, “Foot_Rx” for the right foot. If a foot (left or right) cannot be found nor using its expected label nor using the predefined ones, the block tries to replace it by searching for the lowest point in the input MoCap list (i.e., by selecting the point having the maximum y coordinate if, as usual, the coordinates are referred to a top-left origin). If both feet cannot be found, the block tries to replace them by searching for the two lowest points in the input MoCap list (i.e., by selecting the two points having the maximum y coordinates, always in a top-left coordinate system). If the block cannot find even the lowest or the two lowest points, a warning message is displayed and the output is set as invalid. If the “Use always two lowest” flag is set, the block does not search for the feet coordinates, but instead it directly searches for the two lowest points in the input MoCap list. Note that if the reliability of the MoCap 2D points representing the Center of Gravity or the feet is 0, they will not be considered. The block will continue its search. If no point having reliability greater than 0 can be found the search will fail and the output will be set as invalid. The block returns as output the stability index calculated as the ratio between the height of the body center of gravity with respect to feet and the length of the segment connecting the x projections of the feet or of the two lowest points. Error and warning messages During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output. The

execution phase will fail. Warning: Unable to find the coordinates of the Center of Gravity in the input MoCap data

The block could not find the coordinates of the Center of Gravity in the input MoCap list nor using its expected label nor using the predefined ones. Check if the coordinates of the Center of Gravity are included in your input data. The output will be set as invalid.

Warning: Unable to find the coordinates of the two lowest points in the input MoCap data.

The block could not find the coordinates of the two lowest points in the input MoCap list. This message is displayed if the block failed to find the feet coordinates using both the expected and the predefined labels and it also failed in searching the two points having the maximum (or minimum) y coordinates. Check if the coordinates of the feet are included in your input data. Check also if your input MoCap list contains at least two 2D points. The output will be set as invalid.

Warning: Error in calculating the stability index This message is displayed if the length of the segment connecting the x projections of the feet or of the two lowest points is 0, and therefore the ratio defining the stability index cannot be calculated. Check if your input data are reliable. The output will be set as invalid.


2 Motion tracking

2.1 CentroidsCalc CLSID: 0xF5597E5A,0x3E41,0x43BE,{0xA8,0x83,0x22,0x97,0x44,0x5D,0x83,0x03}

Figure 2.1: The CentroidsCalc block

Inputs Name Datatype Description Input Image Imaging.Image The current input image on which centroids are calculated. It must be a

monochrome image, with a depth of eight or one bit. Usually, it contain a human silhouette.

Outputs Name Datatype Description MoCap Output MoCap.

MoCapDataList A list of MoCap 2D Points containing the x and y coordinates of the calculated centroids. The list contains also a MoCap Scalar, the area of the input blob, and a 2D Bound, the bounding rectangle of the blob.

Parameters The block does not have any parameter. Remarks The CentroidsCalc block computes the position in pixels of some centroids (i.e., barycenters) of some zones of the input blob, assuming that the input image contains an unique blob (connected component). Moreover, the input image must be a monochrome image, with a depth of eight or one bit. The block first computes the bounding rectangle of the input blob, the area (in pixels) that it occupies, and its barycenter. Then, it computes the centroids of other subparts of the blob, whose positions are obtained relatively to the position of the barycenter of the whole blob. Figure 2.2 gives a sketch of the iterations performed by the algorithm.

Figure 2.2: The iterations performed by the centroids algorithm.


The computed points are a rough approximations of the barycenters of the projection of some points (head, shoulders, hands, elbows, knees, and feet) of a human silhouette on a 2D plane, provided that some conditions on the posture of the human being are satisfied:

- the projection of the arms, legs and body must not overlap; - the projection of the arms must be external to the projection of the body (i.e., the arms are assumed to

be between the shoulder and the external sides of the bounding rectangle: an arm kept over the head, for example, fails to satisfy this condition);

- the legs must be under the barycenter of the whole blob. If all the above conditions are satisfied, then the output of the block is a MoCap list containing the following data: Data Type Label Area occupied by the blob (in pixels) MoCap Scalar “Area” Bounding rectangle of the blob MoCap Bound 2D “Bounding_Rect” Position of the baricenter MoCap Point 2D “CoG” Head position MoCap Point 2D “Head” Shoulder left position MoCap Point 2D “Shoulder_Lx” Shoulder right position MoCap Point 2D “Shoulder_Rx” Elbow left position MoCap Point 2D “Elbow_Lx” Elbow right position MoCap Point 2D “Elbow_Rx” Hand left position MoCap Point 2D “Hand_Lx” Hand right position MoCap Point 2D “Hand_Rx” Knee left position MoCap Point 2D “Knee_Lx” Knee right position MoCap Point 2D “Knee_Rx” Foot left position MoCap Point 2D “Foot_Lx” Foot right position MoCap Point 2D “Foot_Rx” Note that all positions are given in pixels, and that left and right refer to the position of the projection, and not to the actual position of that part of the body, i.e., the left hand is the one that, once projected, is on the left side of the image. The centroids coordinates are always given with respect to a top-left origin. Through the custom dialog of the block, it is possible to skip the computation of some points; note that, however, since some points are computed relatively to other ones, there are some dependencies in the computations. For example, if the computation of the position of one shoulder is skipped, the position of the elbow and hand must be skipped too. The custom dialog is shown in Figure 2.3.

Figure 2.3: The custom dialog of the CentroidsCalc block. This block implements the same algorithm implemented by the Imaging.FeatureCalc.CentroidsCalc block. The only difference is that it returns as output a MoCap list instead of a matrix.


Error and warning messages While opening a patch Message Description An error occurred while loading private data. The block could not load its private data. During the init phase Message Description Failed to access input image. The block could not obtain the init data of its input. The input image can be only grayscale (8bpp) or bitonal (1bpp).

The input image must be a monochrome image, with a depth of eight or one bit. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a correct image.

Failed to create ROI. The block could not allocate a ROI (Region of Interest, see the Intel Image Processing Library (IPL) version 2.5 documentation) for the source image. Check if you have enough free memory on your system.

During the execution phase Message Description An error occurred while locking the input. The block could not lock its input. The execution phase will

fail. An error occurred while locking the output. The block could not lock its output. The execution phase

will fail. An error occurred while getting the input ipl image.


2.2 CentroidsDraw CLSID: 0x5AF1AE1E,0xC460,0x4637,{0xB5,0x67,0xCD,0x63,0xEC,0x06,0xE3,0xED}

Figure 2.4: CentroidsDraw block

Inputs Name Datatype Description Input Image (Main)

Imaging.Image The image on which the calculated centroids have to be drawn.

MoCap Input MoCap. MoCapDataList

A list of MoCap 2D Points containing the x and y coordinates of the centroids that have to be displayed. It can also include a MoCap Bound 2D containing the bounding box of the blob whose centroids have been calculated. Usually, it is the output of a CentroidsCalc block.

Outputs Name Datatype Description Output Image Imaging.Image An image consisting of the input image on which the block has drawn

the centroids given as input.


Parameters Name Type Design/Run time Description Draw BRect Boolean Design Time

Run Time Enable/disable drawing of the bounding rectangle. The default value is True.

Draw COGs Boolean Design Time Run Time

Enable/disable drawing of cross markers on the computed centroids. The default value is True.

Lines Combo Design Time Run Time

Type of connecting mesh. Three options are available: “None”, “Central”, and “Hierarchical”. See the Remarks section for further information. The default value is “Hierarchical”.

BRect Color Command Design Time Run Time

When a “BRect Color” command is invoked a dialog box is opened allowing the user to select a color for drawing the bounding rectangle.

COGs Color Command Design Time Run Time

When a “COGs Color” command is invoked a dialog box is opened allowing the user to select a color for drawing cross markers on the computed centroids.

Lines Color Command Design Time Run Time

When a “Lines Color” command is invoked a dialog box is opened allowing the user to select a color for drawing lines connecting the computed centroids.

Remarks The CentroidsDraw block draws on an input image the centroids computed by a CentroidsCalc block. The coordinates of the computed centroids are read from the second input. The image is read from the first input. The “Draw BRect” parameter, if checked, enables the block to draw the bounding rectangle that the CentroidsCalc block computed on the blob whose centroids are displayed. The “Draw COGs” parameter, when checked, enables the block to draw cross markers on the computed centroids. The “Lines” parameter specifies the format of the connecting lines. Three options are currently available: “None” meaning that no lines are drawn, “Central” meaning that the barycenter is connected to all the other points in a star fashion, and “Hierarchical” meaning that a skeleton-like mesh is drawn. The “BRect Color”, “COGs Color” and “Lines Color” commands allow to set the drawing color through a dialog box. Note that if the input MoCap list is not valid, centroids will not be drawn. The block implements the same behavior implemented by the Imaging.Draw.ManParam block, but it receives as input a MoCap list instead of a matrix. Error and warning messages While opening a patch Message Description An error occurred while loading private data. The block could not load its private data. During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. Only 1 or 3 channel images are supported. The input image must have one or three channels. Check

the properties of your input image. You can use a conversion block from the Imaging library to obtain a single channel or three channels image.


execution phase will fail. An error occurred while getting the ipl image. The block could not obtain the output image. The execution

phase will fail.


2.3 CentroidsExtract CLSID: 0x1C8E4F6C,0xA85A,0x4203,{0x99,0x56,0x43,0x9E,0xE7,0xE4,0xA7,0x3B}

Figure 2.5: The CentroidsExtract block

Inputs Name Datatype Description MoCap Input MoCap.

MoCapDataList A list of MoCap 2D Points containing the x and y coordinates of the centroids computed by a previous CentroidCalc block. It can also include a MoCap Bound 2D containing the bounding box of the blob whose centroids have been calculated, and a MoCap Scalar containing the area of the blob. Usually, it is the output of a CentroidsCalc block.

Outputs Name Datatype Description Extracted Value MathDatatype.

Scalar A scalar value extracted from the input MoCap list. It can be the area of the blob whose centroids have been calculated, the top, left, bottom, or right coordinate of its bounding rectangle, the x or y coordinate of a computed centroid.

Parameters Name Type Design/Run time Description Parameter to extract


A combo allowing the user to select which value has to be extracted from the input MoCap list. See the Remarks section for further information.

Remarks The CentroidsExtract block extracts from the MoCap list given as input the values selected through the “Param to extract” parameter. Its input is a list of MoCap 2D Points containing the x and y coordinates of the centroids computed by a previous CentroidCalc block. It can also include a MoCap Bound 2D containing the bounding box of the blob whose centroids have been calculated, and a MoCap Scalar containing the area of the blob. Usually, the input MoCap list is the output of a CentroidsCalc block. The following options are available for the “Param to extract” parameters: Option Extracted Value “Area” Area occupied by the blob (in pixels) “BRectLeft” Top-left point of the bounding rectangle, x coordinate “BrectTop” Top-left point of the bounding rectangle, y coordinate “BRectRight” Bottom-right point of the bounding rectangle, x coordinate “BRectBottom” Bottom-right point of the bounding rectangle, y coordinate “BRectWidth” Width of the bounding rectangle “BRectHeight” Height of the bounding rectangle “CoG_X” Barycenter, x coordinate “CoG_Y” Barycenter, y coordinate “HeadX” Head position, x coordinate “HeadY” Head position, y coordinate “ShoulderLeftX” Shoulder left position, x coordinate “ShoulderLeftY” Shoulder left position, y coordinate “ShoulderRightX” Shoulder right position, x coordinate “ShoulderRightY” Shoulder right position, y coordinate “ElbowLeftX” Elbow left position, x coordinate


“ElbowLeftY” Elbow left position, y coordinate “ElbowRightX” Elbow right position, x coordinate “ElbowRightY” Elbow right position, y coordinate “HandLeftX” Hand left position, x coordinate “HandLeftY” Hand left position, y coordinate “HandRightX” Hand right position, x coordinate “HandRightY” Hand right position, y coordinate “KneeLeftX” Knee left position, x coordinate “KneeLeftY” Knee left position, y coordinate “KneeRightX” Knee right position, x coordinate “KneeRightY” Knee right position, y coordinate “FootLeftX” Foot left position, x coordinate “FootLeftY” Foot left position, y coordinate “FootRightX” Foot right position, x coordinate “FootRightY” Foot right position, y coordinate Notice that, while the computed area, bounding rectangle, and barycenter position are often reliable measures, the calculated centroids can be (roughly) associated to body joints only if the conditions described in the Remarks section of the CentroidsCalc block are satisfied. Otherwise, a computed centroid position should not be considered as a reliable measure of the position of the corresponding body joint. If the selected value cannot be found in the input MoCap list, the output is set as invalid. If the input MoCap list is not valid, the output will be also set as invalid. The coordinate system to which the extracted coordinates are referred depends on the block that produced the input MoCap list: in the case of a CentroidsCalc block, the coordinates are expressed in pixels and referred to a top-left origin. This block implements the same behavior implemented by the Imaging.FeatureCalc.ManParamsExtract block, but it receives as input a MoCap list instead of a matrix. Error and warning messages During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output. The


2.4 CentroidsShow CLSID: 0xDC00AD93,0xDED3,0x4B4A,{0xB3,0xAE,0xEA,0x33,0x92,0x33,0x61,0xED}

Figure 2.6: The CentroidsShow block

Inputs Name Datatype Description MoCap Input MoCap.

MoCapDataList A list of MoCap 2D Points containing the x and y coordinates of the centroids computed by a previous CentroidCalc block. It can also include a MoCap Bound 2D containing the bounding box of the blob whose centroids have been calculated, and a MoCap Scalar containing the area of the blob. Usually, it is the output of a CentroidsCalc block.

Outputs The block does not have any output.


Parameters Name Type Design/Run time Description Disable Output Command Design Time

Run Time Open/close a window in which the computed centroids values are displayed.

Remarks The CentroidsShow block opens a window where a stylized human silhouette is drawn with the superimposed positions of the centroids calculated by a previous CentroidsCalc block. The positions, together with other information such as the occupied area in pixels, and the calculated bounding rectangle, are contained in the MoCap list that has to be provided as input to this block. Such MoCap list is usually provided by a CentroidsCalc block. Notice that, while the computed area, bounding rectangle, and barycenter position are often reliable measures, the calculated centroids can be (roughly) associated to body joints only if the conditions described in the Remarks section of the CentroidsCalc block are satisfied. Otherwise, a computed centroid position should not be considered as a reliable measure of the position of the corresponding body joint. If a value cannot be found in the input MoCap list, it is displayed as invalid in the output window. If the input MoCap list is not valid, the output will be also set as invalid. The coordinate system to which the extracted coordinates are referred depends on the block that produced the input MoCap list: in the case of a CentroidsCalc block, the coordinates are expressed in pixels and referred to a top-left origin. This block implements the same behavior implemented by the Imaging.Output.ManParamsShow block, but it receives as input a MoCap list instead of a matrix. Error and warning messages While opening a patch Message Description An error occurred while loading private data. The block could not load its private data. During the execution phase Message Description An error occurred while locking the input. The block could not lock its input. The execution phase will

fail.

2.5 LKTracker CLSID: 0x799BC775,0x34A8,0x11D5,{0x94,0x33,0xEF,0xF3,0x5C,0xBD,0xD0,0x31}

Figure 2.7: The LKTracker

Inputs Name Datatype Description Input image Imaging.Image The current input image on which feature tracking has to be performed. Outputs Name Datatype Description Output points MoCap.

MoCapDataList A list of MoCap 2D points containing the x and y coordinates of the tracked points.


Parameters Name Type Design/Run time Description Points to track Integer [1, +∞) Design Time The number of features to search for. The output

MoCap list will have a corresponding number of items (MoCap 2D points). The default value is 10.

Lost Points Integer [0, +∞) Design Time The allowed number of lost features. When then number of lost features exceeds the value here declared, the block searches for new features to replace the lost ones. The default value is 1.

Window Size Combo Design Time A performance related parameter of the Lucas - Kanade algorithm. Allowed values are 1, 3, 5, 7, and 9. The default value is 5.

Level Integer [0, 5] Design Time A performance related parameter of the Lucas - Kanade algorithm. The default value is 3.

Distance Threshold

Double [0, +∞) Design Time Run Time

A mechanism to discard tracked features that are still valid but whose positions are not plausible. For each tracked feature, the block calculates the Euclidean distance between its current position and the position it occupied in the previous frame. If this distance is greater than the indicated threshold the feature is considered unreliable.

Enabled Boolean Design Time Run Time

Enable/disable tracking. The default value is True (i.e., tracking enabled).

Refine Boolean Design Time If True, the block refines features locations by using sub-pixel accuracy. The default value is True.

Reset Command Design Time Run Time

When a reset command is invoked the block discards the current tracked features (either lost or still valid), searches for new features, and restarts tracking.

Remarks The LKTracker searches for good features to track in the input image and tracks them using the Lucas - Kanade algorithm. The input image must have one color channel and 8-bit depth. The block returns as output a MoCap list of MoCap 2D points having a number of items equal to the number of currently tracked points. Such number is passed to the block through the “PointsToTrack” parameter. Items are labeled as “Point_X” where X is a progressive number associated to each tracked point and ranging from 0 to (“Points To Track” – 1). Reliability of each output MoCap item is set to 1 if the feature is successfully tracked, and to 0 if the feature is lost. When the number of lost features exceeds “Lost Points”, the block searches for new features to replace the lost ones. Note that if “Lost Points” is zero, the block never replaces lost features. A mechanism to discard tracked features that are still valid but whose positions are not plausible is provided. For each tracked feature, the block calculates the Euclidean distance between its current position and the position it occupied in the previous frame. If this distance is greater than the threshold indicated by the “Distance Threshold” parameter the feature is considered as unreliable. The coordinates of the tracked points are always given with respect to a top-left origin. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The input image must have one color channel. The input image must have one color channel and 8-bit

depth. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a single channel image.

The input image must have a depth of 8. The input image must have one color channel and 8-bit depth. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a 8-bit depth image.

The input image must be gray. The color model of the input image is not correct. Check the properties of your input image. You can use a


conversion block from the Imaging library to obtain the proper color model.





2.6 MatrixLKTracker CLSID: 0x2F527659,0x5422,0x4394,{0x89,0xA9,0xBB,0x50,0xAA,0x5,0x60,0xDA}

Figure 2.8: The MatrixLKTracker

Inputs Name Datatype Description Input image (Main)

Imaging.Image The current input image on which feature tracking has to be performed.

Input Matrix MathDatatype. Matrix

The input matrix containing the initial positions of the points that the tracker should try to track.

Outputs Name Datatype Description Output points MoCap.

MoCapDataList A list of MoCap 2D points containing the x and y coordinates of the tracked points.

Parameters Name Type Design/Run time Description Window Size Combo Design Time A performance related parameter of the Lucas -

Kanade algorithm. Allowed values are 1, 3, 5, 7, and 9. The default value is 5.

Level Integer [0, 5] Design Time A performance related parameter of the Lucas - Kanade algorithm. The default value is 3.

Distance Threshold

Double [0, +∞) Design Time Run Time

A mechanism to discard tracked features that are still valid but whose positions are not plausible. For each tracked feature, the block calculates the Euclidean distance between its current position and the position it occupied in the previous frame. If this distance is greater than the indicated threshold the feature is considered unreliable.

Enabled Boolean Design Time Run Time

Enable/disable tracking. The default value is True (i.e., tracking enabled).

Refine Boolean Design Time If True, the block refines features locations by using sub-pixel accuracy. The default value is True.


When a reset command is invoked the block discards the current tracked features (either lost or


still valid), searches for new features, and restarts tracking.

Remarks The MatrixLKTracker takes as input an image (one color channel, 8-bit depth) and a matrix (floating-point, two columns, an arbitrary number of rows) and attempts to track the features specified in the input matrix using the Lucas - Kanade algorithm. The block returns as output a MoCap list of MoCap 2D points having a number of items equal to the number of currently tracked points (i.e., the number of rows of the input matrix). Items are labeled as “Point_X” where X is a progressive number associated to each tracked point and ranging from 0 to (“Number of rows of the input matrix” – 1). Reliability of each output MoCap item is set to 1 if the feature is successfully tracked, and to 0 if the feature is lost. The block never replaces lost features. A mechanism to discard tracked features that are still valid but whose positions are not plausible is provided. For each tracked feature, the block calculates the Euclidean distance between its current position and the position it occupied in the previous frame. If this distance is greater than the threshold indicated by the “Distance Threshold” parameter the feature is considered as unreliable. Note that the input matrix is read when the block is activated and when the Reset command is invoked. If the input matrix is not valid the last position of the currently tracked points are maintained. If the block was just activated and the input matrix is not valid, the initial position (0, 0) is passed to the tracking algorithm for each point. The coordinates of the tracked points are always given with respect to a top-left origin. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The input image must have one color channel. The input image must have one color channel and 8-bit

depth. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a single channel image.

The input image must have a depth of 8. The input image must have one color channel and 8-bit depth. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain a 8-bit depth image.

The input image must be gray. The color model of the input image is not correct. Check the properties of your input image. You can use a conversion block from the Imaging library to obtain the proper color model.

The input matrix must have 2 columns. The dimensions of the input matrix are not correct. It must have two columns ((x, y) coordinates of the initial points). Check the dimensions of your input matrix.

The input matrix must be a matrix of floats. The input matrix is actually a matrix of integers, but it must be a matrix of floats. You can either use a matrix of floats or a “DomainConv” block to convert a matrix of integers in a matrix of floats.

During the execution phase Message Description An error occurred while locking the input. The block could not lock an input. The execution phase will




2.7 TrackerViewer CLSID: 0xA94AFEA0,0x3957,0x11D5,{0x94,0x33,0x93,0x29,0x9F,0xA0,0xBF,0x17}


Figure 2.9: The TrackerViewer Inputs Name Datatype Description Input Image (Main)

Imaging.Image The image on which the tracked features have to be drawn.

Input MoCap MoCap. MoCapDataList

A list of MoCap 2D points containing the x and y coordinates of the points that have to be displayed.

Outputs Name Datatype Description Output Image Imaging.Image An image consisting of the input image on which the block has drawn

the positions or the trajectories of the given input points. Parameters Name Type Design/Run time Description Mode Combo Design Time

Run Time A combo allowing the user to select the working mode of the block. Two options are currently available: “Draw points”, i.e., each feature is represented as a point drawn on the output image, and “Draw stripes”, i.e., a trajectory composed by the last N positions is drawn for each tracked feature. The default value is “Draw points”.

Color Selection Combo Design Time Run Time

A combo allowing the user to select the mode of color selection. Two options are currently available: “Selected by users”, i.e., users can directly select a desired color by means of a suitable dialog box, and “Controlled within the patch”, i.e., color is determined by the values of the “Frac R”, “Frac G” and “Frac B” parameters (possibly exported and controlled by other blocks). The default value is “Selected by users”.

Points Integer [1, +∞) Design Time Run Time

The number of points (i.e., last positions occupied by a feature) composing the trajectory that has to be drawn. This parameter is enabled only if the “Draw stripes” mode was selected. The default value is 10.

Thickness Integer [1, +∞) Design Time Run Time

The actual thickness of the drawn points or lines. The default value is 1.

Frac R Double [0, 1] Design Time Run Time

The R value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. The default value is 1.

Frac G Double [0, 1] Design Time Run Time

The G value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. The default value is 0.

Frac B Double [0, 1] Design Time Run Time

The B value in the RGB color model. This parameter is enabled only if the “Controlled within the patch” color selection mode was selected. The default value is 0.

Set color Command Design Time Run Time

When a set color command is invoked a dialog box is opened allowing the user to select a color for drawing. This parameter is enabled only if the “Selected by users” color selection mode was


selected. Reset Command Design Time

Run Time When a reset command is invoked the points composing the current trajectories are discarded and the block starts to store and draw new trajectories.

Remarks The TrackerViewer draws markers on the input image, corresponding to the valid 2D points contained in the input MoCap list. Points are valid if their reliability is greater than 0. The block can also draw stripes corresponding to the sequence of segments connecting the last N positions of a valid point, where N is specified by the “Points” parameter. If the input MoCap list is not valid, no points or stripes will be drawn. The color used to draw points and stripes can either be directly selected by users or can be determined within the patch by connecting the output of some blocks to the Frac R, Frac G, and Frac B parameters. The TrackerViewer can be used to display the output of any tracker providing as output a list of MoCap 2D points. Error and warning messages While opening a patch Message Description An error occurred while loading private data. The block could not load its private data. During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. During the execution phase Message Description An error occurred in the locking phase. The block could not lock an input or an output. The

execution phase will fail. An error occurred while getting the output ipl image.

The block could not obtain a pointer to the output ipl image. The execution phase will fail.

Warning: An error occurred while updating the stored trajectories.

The block could not update the stored trajectories. The trajectories will not be drawn. If possible, unlock the patch and restart it.

Warning: An error occurred while drawing trajectories.

The block could not draw a trajectory. In the displayed image some trajectories could be missing. If possible, unlock the patch and restart it.

Warning: An error occurred while drawing points The block could not draw a point. In the displayed image some points could be missing. If possible, unlock the patch and restart it.


3 Posture Recognition

3.1 DeformingNormalizer CLSID: 0xF7E948B6,0x4316,0x11D5,{0x81,0x31,0xE3,0x1F,0x9E,0xBC,0xD7,0x6C}

Figure 3.1: The DeformingNormalizer

Inputs Name Datatype Description In (Main) Imaging.Image The input image containing the object that has to be normalized Bounding Rect MathDatatype.

Matrix A matrix containing the coordinates of the top-left and bottom-right corners of the object’s bounding rectangle. This matrix must be a 2x2 matrix, the first row being the coordinates of the top-left corner, the second row being the coordinates of the bottom-right corner.

Outputs Name Datatype Description Out Imaging.Image The output image in which the scaled bounding rectangle is constrained Scaling Factors MathDatatype.

Matrix A 2x1 vector containing the used scaling factors. Its first item is the scaling factor along x, the second one is the scaling factor along y.

Parameters Name Type Design/Run Time Description Width Integer [1, +∞) Design Time The desired width of the output image. The default

value is 100. Height Integer [1, +∞) Design Time The desired height of the output image. The default

value is 100. Type of Interpolation


The desired type of interpolation for scaling. Available types are “Nearest neighbor”, “Linear”, “Cubic”. The default value is “Nearest neighbor”. For more details see the Intel IPL library documentation.

Remarks The deforming normalizer rescales the bounding rectangle of a detected object (usually a human silhouette) by constraining it within an image having fixed dimensions. Since the scaling factors on the x and y axis are usually different, this operation deforms the object’s silhouette. When used with a human silhouette, it returns, frame-by-frame, the normalized deformed posture. The block makes use of the Intel Image Processing Library (IPL) version 2.5. Each time a new image arrives, a Region of Interest (ROI) is set on the source image corresponding to the object’s bounding rectangle obtained through the second input. Then a scaling operation is performed in order to have the dimensions of the ROI corresponding to the desired dimensions of the output image. Notice that if the input matrix is not valid or one of the dimensions of the current bounding rectangle is 0, the block will scale the whole input image into the output image and the output matrix will be set as invalid. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its


inputs. The input matrix must be a 2x2 matrix. The dimensions of the input matrix containing the

coordinates of the bounding rectangle are not correct. This matrix must be a 2x2 matrix, the first row being the coordinates of the top-left corner, the second row being the coordinates of the bottom-right corner.


execution phase will fail. An error occurred while getting an ipl image. The block could not obtain a pointer to the input or the

output ipl image. The execution phase will fail. An error occurred while allocating an ipl ROI. The block could not allocate the ROI for the source image.

Check if you have enough free memory on your system. The execution phase will fail.

3.2 HuMoments CLSID: 0xA2BEC7D8,0x5387,0x11D5,{0x81,0x31,0xA7,0xC4,0x3D,0x28,0x78,0x6E}

Figure 3.2: The HuMoments block Inputs Name Datatype Description Input Image

Imaging.Image The input image containing the shape whose Hu moments have to be calculated

Outputs Name Datatype Description Hu Moments MathDatatype.

Matrix A vector containing the calculated Hu moments.

Parameters Name Type Design/Run time Description Row Vector Boolean Design Time If True, the output vector will be a row (1x7) vector,

otherwise it will be a column (7x1) vector. The default value is False (so, by default the output is a column vector).

Remarks The block makes use of the Intel OpenCV Library. It appears also in the Imaging.FeatureCalc library. Each time a new image arrives, the block returns a vector containing the Hu moments (Hu, 1962) calculated on the input image. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of its input. The source image must be single channel. In order to calculate Hu moments properly the input image


must be single channel. Check the attributes of your input image. You can use a conversion block from the Imaging library to obtain a single channel image.

During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or output. The execution

phase will fail. An error occurred while getting the input ipl image.


3.3 HuPostureRecognizer CLSID: 0x4A8D5DE1,0x6026,0x11D5,{0x81,0x31,0xCE,0x3E,0x51,0x63,0x6E,0x78}

Figure 3.3: The HuPostureRecognizer Inputs Name Datatype Description Hu Vector (Main) MathDatatype.

Matrix A vector containing the Hu moments calculated on the current input frame. It should be a 7x1 or 1x7 vector of floats (it should be the output of a HuMoments block).

Postures MathDatatype.Matrix

A matrix containing the pre-calculated Hu moments for the candidate postures passed to the recognizer. It should be a Nx7 matrix, where N is the number of candidate postures that you want the block recognizes: each row should contain the seven Hu moments for a given posture.

Outputs Name Datatype Description Recognized Posture


The index of the currently recognized posture. This index refers to a row in the input “Postures” matrix. If the recognizer was not able to recognize any posture, this output is set to invalid.

Confidence MathDatatype. Scalar

A number in [0, 1] describing how much the block is confident to have correctly recognized a given posture.

Parameters Name Type Design/Run time Description Degree Integer (-∞, +∞) Design Time

Run Time The degree of the distance that has to be calculated between the currently input Hu moments and the Hu moments associated to each posture in the “Postures” matrix. The default value is 2. See the “Remarks” section for a more complete explanation.

N Integer [1, +∞) Design Time The number of previous inputs that has to be considered in order to perform recognition. The default value is 10. See the “Remarks” section for a more complete explanation.

Threshold Integer [0,100] Design Time Run Time

A threshold used in the recognition process. The default value is 50. See the “Remarks” section for a more complete explanation.

Recognize Posture

Command Design Time Run Time

Recognition is performed each time a “Recognize Posture” command arrives. Otherwise, the block


updates its internal data structures, but no recognition is performed and an invalid output is returned. This parameter is exported by default.

Remarks This block tries to recognize a posture by associating it to one of a set of candidate postures. The selected candidate posture is chosen by minimizing the vector distance between the Hu moments (Hu, 1962) calculated on the current input posture and the Hu moments pre-calculated on each candidate posture. The Hu moments for the current input posture can be calculated at run-time by a HuMoments block. The Hu moments for each candidate posture should be calculated in advance (eventually using an ad hoc patch containing a HuMoments block) and stored in a matrix provided as input to this block. Such matrix must contain a row for each candidate posture having as values the associated seven Hu moments. Each time a new Hu vector associated to the current input posture arrives, if the “RecognizePosture” command was given, the block firstly calculates the vector distance between the input Hu vector and each row in the “Postures” input matrix. Distance is calculated accordingly to the following formula:

pn

k

p

kkp vuvud1

1

),(

−= ∑

=

where u and v are vector having n elements, and p is the degree of the distance.

The degree p of the distance can be set through the “Degree” parameter and can be changed at run-time. By default p is set equal to 2, corresponding to the standard Euclidean distance. When p = 1, the 1-distance is

obtained: ∑=

−=n

kkk vuvud

11 ),( . When p ≤ 0, the block calculates the ∞-distance, that is kknk

vuvu −==∞ max),(

..1d .

Then, the candidate posture corresponding to the minimum distance is selected and its index is stored in an internal data structure. In order to have a quite stable output, at this point the block looks at the last N recognized postures (where N is provided as design-time parameter to the block) and compares them against the threshold provided as parameter as well. The threshold represents a percentage of recognition during the last N recognitions: if the most frequently recognized posture between the last N postures has a percentage of recognition above this threshold, such a posture is recognized, otherwise the output is set as invalid. For instance, if the threshold is set to 90% and N is set to 10, posture 3 will be recognized only if the array of the indexes of the last 10 recognized postures contains the index 3 at least nine times. When the current input is invalid or the “RecognizePosture” command was not given, the block fills its internal array with -1. In this way, the recognition “history”, contained in the internal array, refers only to phases when a “RecognizePosture” command arrives. Of course, the “RecognizePosture” command should be given for a time interval long enough to allow the block to perform recognition, given the desired value of N. Notice that postures cannot be recognized if their time duration is too short with respect to the values of N , Threshold, and the sampling frequency. For instance, if N = 10, Threshold = 100% , and the frame rate is 25 Hz, a posture can be recognized only if its duration is longer than 10x(1/25) = 0.4 s. The block returns also a confidence index in the range [0, 1] describing how much it is confident to have correctly recognized a given posture. The confidence index is determined by comparing the value of the calculated minimum distance with the distance immediately larger than the minimum one, similar values of the two distances meaning an ambiguous recognition. Notice that if one or both the current inputs are not valid, the block will update its internal data structures, but the output will be set as invalid. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. The input matrix must have 7 columns. The “Postures” matrix must be a Nx7 matrix, being N the

number of postures composing the database of the candidate postures. Each row must contain the seven Hu moments of a candidate posture. Check the dimensions of your input matrix and verify its format.

The input vector must have 7 items. The input vector contains the Hu moments calculated on the current frame. It must be a 7x1 or 1x7 vector. It should be the output of a HuMoments block. Check the dimensions of your input vector.

The input matrix must be a matrix of floats. The input matrix contains the Hu moments for the


candidate postures: it must be a matrix of floats. You can either use a matrix of floats or a “DomainConv” block to convert a matrix of integers in a matrix of floats.

The input vector must be a vector of floats. The input vector contains the Hu moments calculated on the current frame: it must be a vector of floats. You can either use a vector of floats or a “DomainConv” block to convert a vector of integers in a vector of floats.


fail. An error occurred while locking the output. The block could not lock the output. The execution phase

will fail.

3.4 NotDeformingNormalizer CLSID: 0xDCF04C4E,0xF874,0x4897,{0x87,0x78,0x62,0x2D,0x10,0xEF,0x18,0xBD}

Figure 3.4: The NotDeformingNormalizer

Inputs Name Datatype Description In (Main) Imaging.Image The input image containing the object that has to be normalized Bounding Rect MathDatatype.

Matrix A matrix containing the coordinates of the top-left and bottom-right corners of the object’s bounding rectangle. This matrix must be a 2x2 matrix, the first row being the coordinates of the top-left corner, the second row being the coordinates of the bottom-right corner.

Outputs Name Datatype Description Out Imaging.Image The output image in which the scaled bounding rectangle is placed Normalized on Width


“Normalized on Width” is set to 0 if the bounding rectangle was normalized with respect to its height, and it is set to 1 if the bounding rectangle was normalized with respect to its width.

Scaling Factor MathDatatype. Scalar

The used scaling factor. The scaling factor is the same both along x and along y.

Parameters Name Type Design/Run time Description Width Integer [1, +∞) Design Time The desired width of the output image. The default


value is 288. Normalized Rectangle Width

Integer [1, +∞) Design Time Run Time

The desired width of the object’s bounding rectangle in the output image. If normalization is performed on width, the actual width of the object’s bounding rectangle will be scaled to this value. The default value is 100.


Normalized Rectangle Height

Integer [1, +∞) Design Time Run Time

The desired width of the object’s bounding rectangle in the output image. If normalization is performed on width, the actual width of the object’s bounding rectangle will be scaled to this value. The default value is 100.

Type of Interpolation

Combo Design Time Run time

The desired type of interpolation for scaling. Available types are “Nearest neighbor”, “Linear”, “Cubic”. The default value is “Nearest neighbor”. For more details see the Intel IPL library documentation.

Force Normalization Direction


If True, the normalization direction can be forced to be always the same. Otherwise, the block will decide, frame-by-frame, which is the best direction along which normalization should be performed. The default value is False.

Forced Direction


This combo is enabled only if the “Force Normalization Direction” flag is set. Its values are “Width” and “Height” and it allows the user to select the direction along which normalization has to be forced. The default value is “Width”.

Remarks The non-deforming normalizer scales the bounding rectangle of a detected object (usually a human silhouette) and places it at the center of the output image. Depending on the relationship between width and height of the output image and width and height of the current bounding rectangle a decision is made about the direction along which scaling is performed: in each frame the bounding rectangle is scaled either with respect to its height (i.e., constraining the height to a fixed dimension) or with respect to its width (i.e., constraining the width to a fixed dimension). Scaling along the remaining direction is performed using the same factor. Therefore, normalization does not deform the original shape of the detected object. A flag is available to force normalization to be performed always along the same direction, but note that, in this case, it can happen to have the rescaled bounding rectangle larger than the output image. In such a case, the normalized silhouette cannot be displayed. The block makes use of the Intel Image Processing Library (IPL) version 2.5. Each time a new image arrives, a Region of Interest (ROI) is set on the source image corresponding to the object’s bounding rectangle obtained through the second input. Another ROI is set at the center of the destination image having dimensions corresponding to the desired normalized dimensions. Then a scaling operation is performed in order to have the dimensions of the input ROI corresponding to the desired dimensions of the output ROI. Notice that if the input matrix is not valid or one of the dimensions of the current bounding rectangle is 0, the block will scale the whole input image into the output image, and the scalar outputs will be set as invalid. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. The input matrix must be a 2x2 matrix. The dimensions of the input matrix containing the

coordinates of the bounding rectangle are not correct. This matrix must be a 2x2 matrix, the first row being the coordinates of the top-left corner, the second row being the coordinates of the bottom-right corner.


execution phase will fail. An error occurred while getting an ipl image. The block could not obtain a pointer to the input or the

output ipl image. The execution phase will fail. Invalid rescaling dimensions. One or both the desired dimensions of the normalized


bounding rectangle are larger than the dimensions of the output image. The block will scale the whole input image into the output image. The scalar outputs will be set as invalid. Check the dimension you passed as parameters.

An error occurred while allocating an ipl ROI. The block could not allocate the ROI for the source image. Check if you have enough free memory on your system. The execution phase will fail.

Warning: The rescaled width is larger than the width of the output image.

Normalization was forced along height, but the rescaled width is larger than the width of the output image. The normalized silhouette cannot be displayed.

Warning: The rescaled height is larger than the height of the output image.

Normalization was forced along width, but the rescaled height is larger than the height of the output image. The normalized silhouette cannot be displayed.

3.5 SPMPostureRecognizer CLSID: 0x8E4BA501,0x58D8,0x11D5,{0x8F,0x58,0x00,0x50,0xDA,0xE0,0x41,0x33}

Figure 3.5: The SPMPostureRecognizer

Inputs Name Datatype Description Input Image Imaging.Image The current input image. The input image should be comparable with the

images constituting the current collection of postures, i.e., if the collection is composed by images containing a binarized and normalized silhouette, the input image should have a similar content.

Outputs Name Datatype Description Recognized Posture


The index of the currently recognized posture. This index refers to the image in the current collection of postures that was judged to be the most similar with respect to the actual input image. If the recognizer was not able to recognize any posture this output is set to invalid.

Parameters Name Type Design/Run time Description Postures File name Design Time The name of a bitmap file in the folder containing

the bitmap files constituting the collection of postures that the block has to consider during its processing. Note that every posture should be saved in a bitmap file contained in the same folder.

Threshold Double [1, +∞) Design Time Run Time

A threshold used in the recognition process. The default value is 1000. See the “Remarks” section for a more complete explanation.

Remarks The SPMPostureRecognizer identifies a posture in the current input image by comparing it with the postures contained in a collection of images, and returns the index of the recognized posture with respect to the considered collection of images. Postures should be saved in a collection of bitmaps, each one contained in the same folder. The input images should be comparable with the images in the collection, i.e., they should have the same format and content. For


example, if the collection is composed by black and white images containing a binarized and normalized silhouette, the input images should also be black and white images and they should have a similar content. In order to recognize a posture, the block uses a simple pattern matching technique. Firstly, it calculates the logical and between the current input image and the images contained in the collection, then it applies the threshold passed as parameter on the area of the resulting image. A collection of four postures extracted from the sample microdance distributed with EyesWeb 2.5 is included in the EyesWeb Motion Analysis library distribution. Note that the bitmaps contained in such collection are rescaled using the NotDeformingPostureNormalizer. Error and warning messages During the init phase Message Description The given path to a folder containing the images of the postures is not valid.

The path that was passed to the block by means of the “Postures” parameter is not valid, i.e., the block could not find the specified bmp file. Check if the file name you passed to the block is correct and if this file exists.






4 SpaceAnalysis

4.1 HitCellDetector CLSID: 0x4461FDAE,0xF54C,0x4404,{0x82,0xFA,0x4D,0x81,0xD1,0x37,0x4E,0xD2}

Figure 4.1: The HitCellDetector

Inputs Name Datatype Description Position MathDatatype.

Matrix The current position of a tracked point in a bounded virtual 2D space. The range of each dimension in the input 2D space is [-1, 1], so the values contained in the input vector should be in the range [-1, 1].

Outputs Name Datatype Description Hit Cell MathDatatype.

Matrix A 1x2 matrix containing the indexes (h, k) of the hit cell, i.e. the cell currently occupied by the input point in the input space.

Parameters Name Type Design/Run Time Description Cells along x Integer [1, +∞) Design Time The desired number of cells along the x axis. It is used

to build the grid in the init phase. The default value is 1. Cells along y Integer [1, +∞) Design Time The desired number of cells along the y axis. It is used

to build the grid in the init phase. The default value is 1. Remarks The HitCellDetector block builds a grid having a specified number of cells and superimposes it to the input 2D space. This is a bounded space ranging in [-1, 1] on each dimension. Then, each time a new position of a tracked point arrives, the block returns the indexes (h, k) of the hit cell, i.e., of the cell currently occupied by the tracked point. Note that all the cells in the grid have the same dimensions. If different resolutions are needed in different parts of the input space, more HitCellDetector blocks can be used, working on portions of the input space. Of course, in this case the coordinates of the tracked point have to be referred to the considered portion of the input space. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. The input position must be a vector of integers. The input position is actually a vector of floats, but it must

be a vector of integers. You can either use a vector of integers or a “DomainConv” block to convert a vector of float in a vector of integers.

The input position must be a [1x2] vector. The dimensions of the input vector are wrong: it must be a 1x2 vector (the first element being the x coordinate, and the second one the y coordinate)




will fail. The current input is out of range. The current position of the tracked point results to be

outside the input space. Such position must be in the range [-1, 1] both along x and y. The output matrix will be set as invalid.

Note that if the input matrix is not valid, the output matrix will be also set as invalid.

4.2 ImageHitCellDetector CLSID: 0x4461FDAE,0xF54C,0x4404,{0x82,0xFA,0x4D,0x81,0xD1,0x37,0x4E,0xD2}

Figure 4.2: The ImageHitCellDetector Inputs Name Datatype Description Position (Main) MathDatatype.

Matrix The current position (in pixels) of a tracked point.

Source Image Imaging.Image The reference image on which the grid is superimposed. It is used to calculate the dimensions of each cell.

Outputs Name Datatype Description Hit Cell MathDatatype.

Matrix A 1x2 matrix containing the indexes (h, k) of the hit cell, i.e., the cell currently occupied by the input point.

Parameters Name Type Design/Run Time Description Cells along x Integer [1, +∞) Design Time The desired number of cells along the x axis. It is used

to build the grid in the init phase. The default value is 1. Cells along y Integer [1, +∞) Design Time The desired number of cells along the y axis. It is used

to build the grid in the init phase. The default value is 1. Remarks The ImageHitCellDetector block builds a grid having a specified number of cells and superimposes it to the input image. Then, each time a new position (expressed in pixels) of a tracked point arrives, it returns the indexes (h, k) of the hit cell, i.e., of the cell currently occupied by the tracked point. Note that all the cells in the grid have the same dimensions. If different resolutions are needed in different parts of the image, more ImageHitCellDetector blocks can be used, working on subimages. Of course, in this case the coordinates in pixels of the tracked point have to be referred to the subimage.


Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. The number of cells is larger than the size of the input image.

The number of cells specified as parameter is larger than the number of pixels in the input image. Check the value of the “Cells along x” and “Cells along y” parameters.

The input position must be a vector of integers. The input position is actually a vector of floats, but it must be a vector of integers. You can either use a vector of integers or a “DomainConv” block to convert a vector of float in a vector of integers.

The input position must be a [1x2] vector. The dimensions of the input vector are wrong: it must be a 1x2 vector (the first element being the x coordinate, and the second one the y coordinate)



will fail. The current input is out of range. The current position of the tracked point results to be

outside the input image. The output matrix will be set as invalid.

Note that if the input matrix is not valid, the output matrix will be also set as invalid.

4.3 PositionDependingPotentials CLSID: 0xBF11016C,0x3D48,0x11D5,{0x8F,0x57,0x0,0x50,0xDA,0xE0,0x41,0x33}

Figure 4.3: The PositionDependingPotentials block Inputs Name Datatype Description Hit cell (Main) MathDatatype.

Matrix The indexes (h, k) of the current hit cell. Usually, this input comes from an “HitCellDetector” block or an “ImageHitCellDetector” block.

Initialization matrix

MathDatatype.Matrix

A matrix containing the initialization values for the potential function. The number of elements in this matrix must be equal to the number of cells. Therefore, the number of rows must be equal to the number of cells along y and the number of columns must be equal to the number of cells along x. This input is not required for the execution. If it is not provided or the matrix is invalid, the potential function is initialized with zeros.

Outputs Name Datatype Description Potential function MathDatatype.

Matrix The matrix representing the calculated potential function. The number of elements in this matrix is equal to the number of cells. Therefore, the


number of rows is equal to the number of cells along y and the number of columns is equal to the number of cells along x.

Parameters Name Type Design/Run time Description Cells along x Integer [1, +∞) Design Time The number of cells in the grid along the x axis. This

parameter should have the same value than the corresponding one in the “HitCellDetector” or “ImageHitCellDetector” block providing the input to this block. The default value is 1.

Cells along y Integer [1, +∞) Design Time The number of cells in the grid along the y axis. This parameter should have the same value than the corresponding one in the “HitCellDetector” block or “ImageHitCellDetector” providing the input to this block. The default value is 1.

Read from initialization input


If True, the potential function is initialized using the values contained in “Initialization matrix”, else the potential function is initialized with zeros. The default value is False.


Each time a reset command arrives, the potential function is reinitialized: the new initial values are the one in the “Initialization matrix” or zeros, depending on the “Read from initialization input” flag.

Remarks This block calculates the values φij of a potential function directly depending on the current position of a tracked point: φij = φij (h, k) where (h, k) is the cell currently occupied by the point. In this way it is possible to define potential functions moving in the space together with the movement of a tracked point. Each time the indexes of the currently hit cell arrive, the block updates the output matrix, containing the values the potential function has in correspondence with the cells of the defined grid. Such matrix can be initialized with zeros of with values provided through the “Initialization matrix”, depending on the “Read from initialization input” flag. For example, the initial values could be predisposed and saved in a text file. If the initialization matrix is not provided or it is not valid, the potential function is also initialized with a matrix of zeros. A reset command is available in order to reinitialize the output matrix while executing a patch. The desired potential function can be selected using the custom dialog in the picture below (Figure 4.4).

Figure 4.4


The combo box “Type of function” allows the user to select the desired potential function. Three kinds of potentials function are currently available:

- Increment/Decrement: this potential function measures a sort of “activity level” of each cell. Each time a cell is hit the corresponding value of the potential function is incremented using the increment factor. The values corresponding to the remaining cells (that were not hit) are decremented using the decrement factor. Note that the decrement factor must be a negative number to decrement properly. Note also that, at the moment, only this kind of simple “linear” increment/decrement mechanism is available. If the decrement factor is set to zero, the cells that are not hit will maintain their previous activity level.

- Polynomial: this option corresponds to the following potential function:

( )

=

≠−+−=

),(),( if 1

),(),( if 1

),(

khji

khjikjhikh N

KNN

ijφ

When K = 1, it corresponds to the ratio of 1 with the distance between a cell and the hit cell. So the potential function decreases, when the distance from the hit cell increases. The user can select the distance to be used by setting N. When N = 2, the standard Euclidean distance will be used. By setting K potential functions depending on the ratio of 1 with powers of the distance can be obtained. For example if N = 2 and K = 2, a potential function decreasing with the square of the Euclidean distance is obtained. By default the Euclidean distance is used (i.e., N = 2 and K = 1).

- Occupation Rates: in this case, the potential function matrix contains the occupation rate of each cell, that is the ratio between how many times a cell was hit and how many cells were it until the current time instant. By default, the occupation rates are provided as float numbers in the range [0, 1]. If the “In percentage” flag is set, they are provided as percentages, that is in the range [0, 100].


inputs. The vector containing the hit cell must be a vector of integers.

The vector containing the indexes (h, k) of the hit cell is actually a vector of floats, but it must be a vector of integers. You can either use a vector of integers or a “DomainConv” block to convert a vector of floats in a vector of integers.

The vector containing the hit cell must be a [1x2] vector.

The dimensions of the input vector are wrong: it must be a 1x2 vector (the first element being the h index of the hit cell, and the second one the k index)

The initialization matrix must be a [m x n] matrix. The dimensions of the initialization matrix are wrong: the initialization matrix must have a number of rows equal to the number of cells along y and a number of columns equal to the number of cells along x. When the error message is displayed, m and n are replaced by the expected correct dimensions of the initialization matrix.

Warning: The initialization matrix is not connected: the potential function will be initialized with zeros.

Even if the “Read from initialization input” flag is set, the input matrix is not available (its pin is not connected). The potential function will be initialized with zeros.



will fail. The current input cell is out of range: h = H , k = K.

The current hit cell results to be outside the grid. When the error message is displayed, H and K are replaced by the


resulting wrong values of the indexes h and k. The output matrix will be set as invalid.

Note that if the block is initializing the potential function using the values contained in the initialization matrix the “Reading the initialization matrix...” is displayed, otherwise the block displays “Initializing with a null matrix...”. “Initializing with a null matrix...” is displayed also if the “Read from initialization input” flag is set, but the initialization matrix is not available. Note also that if the input matrix is not valid, the output matrix will be also set as invalid.

4.4 RegionDependingPotentials CLSID: 0x1F36DF3F,0xE4F7,0x43CC,{0xA2,0x82,0x84,0xB2,0xC6,0x6B,0x1F,0xD3}

Figure 4.5: The RegionDependingPotentials block Inputs Name Datatype Description Hit cell (Main) MathDatatype.

Matrix The indexes (h, k) of the current hit cell. Usually, this input comes from an “HitCellDetector” block or an “ImageHitCellDetector” block.

Defined region MathDatatype.Matrix

A matrix defining which cells belong to the region. The number of elements in this matrix must be equal to the number of cells. Therefore, the number of rows must be equal to the number of cells along y and the number of columns must be equal to the number of cells along x. For each element of the matrix, if the value of an element is equal to zero, then the corresponding cell belongs to the defined region, otherwise the corresponding cell does not belong to the region.

Initialization matrix

MathDatatype.Matrix

A matrix containing the initialization values for the potential function. The number of elements in this matrix must be equal to the number of cells. Therefore, the number of rows must be equal to the number of cells along y and the number of columns must be equal to the number of cells along x. This input is not required for the execution. If it is not provided or the matrix is invalid, the potential function is initialized with zeros.

Outputs Name Datatype Description Potential function MathDatatype.

Matrix The matrix representing the calculated potential function. The number of elements in this matrix is equal to the number of cells. Therefore, the number of rows is equal to the number of cells along y and the number of columns is equal to the number of cells along x.

Is Hit MathDatatype.Scalar

This output is set to 1 if the current hit cell (whose indexes are the current “Hit cell” input) belongs to the defined region, otherwise it is set to 0.

Parameters Name Type Design/Run time Description Cells along x Integer [1, +∞) Design Time The number of cells in the grid along the x-axis.

This parameter should have the same value than the corresponding one in the “HitCellDetector” or “ImageHitCellDetector” block providing the input to this block. The default value is 1.

Cells along y Integer [1, +∞) Design Time The number of cells in the grid along the y-axis. This parameter should have the same value than the corresponding one in the “HitCellDetector” or


“ImageHitCellDetector” block providing the input to this block. The default value is 1.

Read from initialization input


If True, the potential function is initialized using the values contained in “Initialization matrix”, else the potential function is initialized with zeros. The default value is False.


Each time a reset command arrives, the potential function is reinitialized: the new initial values are the one in the “Initialization matrix” or zeros, depending on the “Read from initialization input” flag.

Remarks This block calculates the values φij of a potential function directly depending on the definition of a region in the space. φij = φij (R(i,j)) where R(i,j) is the region to which the cell (i, j) belongs. The “Defined region” matrix characterizes the defined region: therefore, space results divided up in two regions. A set of cells will constitute the region defined by the “Defined region” matrix. The remaining cells will constitute in fact another region: the region of the cells that do not belong to defined region. Each time a new input arrives, i.e., each time the block reads what cell is currently occupied by the tracked point a hit function is applied to the cells belonging to the hit region (either the defined region or the set of cells not belonging to the defined region). A miss function is applied to the remaining cells. The hit and miss function currently implemented can be selecting using the custom dialog in the picture below (Figure 4.6).

Figure 4.6 The combo box “Type of function” allows the user to select the desired hit and miss functions. Two kinds of hit and miss functions are currently available:

- Increment/Decrement: these hit and miss functions can be used to measure a sort of “activity level” of each region. Each time a cell is hit, a check is done to verify if the cell belongs to the region. If the test is positive, the activity level for all the cells belonging to the region is increased using the increment factor (i.e., the hit function is a linear increment). The values corresponding to the remaining cells (that do not belong to the region) are decremented using the decrement factor (i.e., the miss function is a linear decrement). If the test is negative, the activity level for all the cells belonging to the region is decremented using the decrement factor, while the values corresponding to the remaining cells (that do not belong to the region) are incremented using the increment factor. Note that the decrement factor must be a negative number to decrement properly. Note also that, at the moment, only this kind of simple linear increment/decrement mechanism is available. If the decrement factor is set to zero, the cells that do not belong to the hit region (the defined one or the other one) will maintain their previous activity level: in fact, the miss function is constantly set to zero.


- Occupation Rates: in this case, the potential function matrix contains the occupation rates of the defined region or of the cells not belonging to the region. The occupation rate of the defined region will be associated in the potential matrix to all the cells belonging to the region. The same will happen for the cells that do not belong to the defined region (that in fact belong to the other implicit region). In this case, the hit and miss functions are respectively an increment and a decrement of the occupation rates. By default the occupation rates are provided as float numbers in the range [0, 1]. If the “In percentage” flag is set, they are provided as percentages, i.e., in the range [0, 100].


inputs. The vector containing the hit cell must be a vector of integers.

The vector containing the indexes (h, k) of the hit cell is actually a vector of floats, but it must be a vector of integers. You can either use a vector of integers or a “DomainConv” block to convert a vector of floats in a vector of integers.

The vector containing the hit cell must be a [1x2] vector.

The dimensions of the input vector are wrong: it must be a 1x2 vector (the first element being the h index of the hit cell, and the second one the k index)

The region must be defined using a matrix of integers.

The matrix used to define the region is actually a matrix of floats, but it must be a matrix of integers. You can either use a matrix of integers or a “DomainConv” block to convert a matrix of floats in a matrix of integers.

The region must be defined using a [m x n] matrix.

The dimensions of the matrix defining the region are wrong: this matrix must have a number of rows equal to the number of cells along y and a number of columns equal to the number of cells along x. When the error message is displayed, m and n are replaced by the expected correct dimensions of the matrix.

The initialization matrix must be a [m x n] matrix. The dimensions of the initialization matrix are wrong: the initialization matrix must have a number of rows equal to the number of cells along y and a number of columns equal to the number of cells along x. When the error message is displayed, m and n are replaced by the expected correct dimensions of the initialization matrix.

Warning: The initialization matrix is not connected: the potential function will be initialized with zeros.

Even if the “Read from initialization input” flag is set, the input matrix is not available (its pin is not connected). The potential function will be initialized with zeros.



will fail. The current input cell is out of range: h = H , k = K.

The current hit cell results to be outside the grid. When the error message is displayed, H and K are replaced by the resulting wrong values of the indexes h and k. The output matrix will be set as invalid.

Note that if the block is initializing the potential function using the values contained in the initialization matrix the “Reading the initialization matrix...” is displayed, otherwise the block displays “Initializing with a null matrix...”. “Initializing with a null matrix...” is displayed also if the “Read from initialization input” flag is set, but the initialization matrix is not available. The “Reading region...” message is displayed when the block is reading the matrix defining the region. Note also that if the input matrix is not valid, the output matrix will be also set as invalid.


4.5 SpaceAnalysisViewer CLSID: 0x59EB3285,0xD461,0x11D5,{0x81,0x31,0xF1,0xEB,0xAC,0x7B,0x12,0x0C}

Figure 4.7: The SpaceAnalysisViewer

Inputs Name Datatype Description Input Matrix

MathDatatype.Matrix

The matrix that has to be displayed. In particular, it can be the output of a PositionDependingPotentials or of a RegionDependingPotentials block

Outputs Name Datatype Description Output Image Imaging.Image An image whose colors are proportionally dependent on the values of the

items of the input matrix. Parameters Name Type Design/Run Time Description Width Integer [1, +∞) Design Time The desired width of the output image. The default


value is 288. Min Double

(-∞, +∞) Design Time Run Time

The minimum value an item in the input matrix can assume. This value is used for the linear rescaling of the input values into the values of the color coordinates used in the output image. The default value is -1.

Max Double (-∞, +∞)


The maximum value an item in the input matrix can assume. This value is used for the linear rescaling of the input values into the values of the color coordinates used in the output image. The default value is 1.

Draw Grid Boolean Design Time Run Time

If this flag is set to True a grid is drawn on the output image showing the boundaries of the rectangular regions of the output image associated with each element of the input matrix. The default value is False.

AutoArrange Boolean Design Time If this flag is set to true the output image is resized with respect to the desired width and height in order to be able to divide it into a number of regions having the same size that are then associated with the items of the input image. The default value is False.

Mode Combo Design Time Run Time

A combo box allowing the user to select which color parameter has to be associated with the items of the input matrix. Since the block uses the HSV color model, the possible options are “Map matrix on H”, “Map matrix on S”, and “Map matrix on V”. The default option is “Map matrix on S”.

H Integer [0, 359] Design Time Run Time

The desired value for H in the HSV color model. This parameter is disabled when the “Map matrix on H” is selected. The default value is 0.

S Double [0, 1] Design Time Run Time

The desired value for S in the HSV color model. This parameter is disabled when the “Map matrix on S” is selected. The default value is 1.

V Double [0, 1] Design Time Run Time

The desired value for V in the HSV color model. This parameter is disabled when the “Map matrix on V” is


selected. The default value is 1. Remarks The SpaceAnalysisViewer displays the matrix it receives as input by producing an image composed by a number of rectangles equal to the number of items in the input matrix, each one colored depending on the value of the corresponding item in the input matrix. The values of the items in the input matrix can be associated with the H, the S, or the V value in the HSV color space (through the Mode parameter). A linear mapping between the values in the input matrix and the selected color parameter is performed. Notice that, even if included in the Motion Analysis library, the block can be used to have a visualization of a generic matrix as well. Error and warning messages During the init phase Message Description An error occurred while getting the init data. The block could not obtain the init data of some of its

inputs. During the execution phase Message Description An error occurred in the locking phase. The block could not lock its input or its output. The

execution phase will fail. An error occurred while getting the output ipl image.

The block could not obtain a pointer to the output ipl image. The execution phase will fail.

Note that if the current input is not valid, the output will not be generated.


5 References Argyle M., Bodily Communication, Methuen&Co Ltd, London, 1980. Camurri A., Trocca R., Volpe G., Interactive Systems Design: A KANSEI-based Approach. Proc. NIME 2002, Dublin, May 2002. Camurri A., De Poli G., Leman M., and Volpe G., A multi-layered conceptual framework for expressive gesture applications, in Proc. Workshop on Current Research Directions in Computer Music, Barcelona, November 2001. Camurri A., Mazzarino B., Trocca R., Volpe G., Real-Time Analysis of Expressive Cues in Human Movement. Proc. CAST01, GMD, St.Augustin-Bonn, September 2001. Hu M.K., Visual pattern recognition by moment invariants. IRE Trans. Information Theory, vol. IT-8, pp. 179—187, Feb. 1962. Laban R., Modern Educational Dance, Macdonald & Evans Ltd. London, 1963. Laban R., Lawrence F.C., Effort, Macdonald&Evans Ltd. London, 1947. Wallbott H.G., The measurement of Human Expressions, in Walbunga von Rallfer-Engel, “Aspects of communications”, pp. 203-228, 1980. Intel Open Computer Vision Library, Reference Manual. Available at: http://www.intel.com/research/mrl/research/opencv/ and http://sourceforge.net/projects/opencvlibrary/ Intel Image Processing Library (IPL), Reference Manual. Available at: http://developer.intel.com/software/products/perflib/ipl/

http://www.intel.com/research/mrl/research/opencv/

http://sourceforge.net/projects/opencvlibrary/

http://developer.intel.com/software/products/perflib/ipl/

N u m e n S o f t S.n.c. di Massimiliano Peri e Andrea Ricci

BlobAnalysis Library The BlobAnalysis Library is contained in BlobAnalysis.dll. This library is based on the MoCapData datatype, contained in MoCapDatatype.dll. This library provides a set of blocks to extract multiple pixel concentrations (blobs) from a grayscale image, to track these regions and to identify them during time, to extract several features from each region. These blocks are placed in the “Imaging.BlobAnalysis” EyesWeb object hierarchy.

1. ExtractRegions CLSID: {185FE40E-B85E-4891-825C-4B9DB7905D57}

Inputs

GRAYSCALE_IMAGE Image The input grayscale image used to identify the regions

Outputs

BLOB_BRECTS MoCapDataList The calculated area bounding rectangles QUANTIZED_IMAGE Image The quantized image used to identify the occupied areas

Parameters

NumRegions Integer The number of maximum regions to extract. If more than NumRegions are identified, the bigger ones are chosen

QScaleFactor Integer number of times to reduce the image for quantization QThresh Integer treshold value applied to the reduced quantized image QCloseIterations Integer number of times the close algorithm is applied to the

tresholded image QOutput Boolean Toggle the quantized image output

Remarks

Itentify multiple regions, calculating their bounding rectangle approximation. The regions are calculated in a top left first order. To track and identify the regions use the TrackRegions block linked to the output of this block. Every identified region is stored in a CMoCapBound2D item, with coordinates relative to the image pixels.

2. TrackRegions CLSID: {DB5C9C6F-91D7-4232-82BB-F0A57B719445}

Inputs

MOCAP_INPUT MoCapDataList The regions to track POINT_MATRIX Matrix You can link this input to a DisplayClick and click over

the tracked regions to swap them and override the tracker decisions.

Outputs

MOCAP_OUTPUT MoCapDataList The tracked regions

Parameters

Reset Command Set the tracker to its initial state

Remarks

The block tracks and identify multiple regions, extracted by the ExtractRegions block. To configure this block and add new tracked regions, use its custom dialog. In the initial state, the regions are calculated in a top left first order.

3. FeatureExtract CLSID: {28EE2C4A-7604-457C-B875-492561A38EF3}

Inputs

AnalysisImage Image <Notes> MoCapBounds MoCapDataList <Notes>

Outputs

MoCapData MoCapDataList <Notes>

Parameters

Area Boolean Calculate the area of each region Baricenter Boolean Calculate the baricenter of each region Primary_Axis Boolean Calculate the primary axis of each region Secondary_Axis Boolean Calculate the secondary axis of each region Label_Prefix String The prefix to add to every region label

Remarks

This block calculates baricenters, area and orientation of every region represented by MoCapBound2D items in the MoCapDataList input.

N u m e n S o f t S.n.c. di Massimiliano Peri e Andrea Ricci

MoCap Library The MoCap Library is contained in the MoCapHelpers.dll and MoCapDatatype.dll DLL files.

1. MoCap.Filters.PositionKalmanFilter CLSID: {{2C92FD32-4866-4796-BF5B-4FA2ABB691C2}

Inputs

MoCap List MoCapDataList

Outputs

MoCap List MoCapDataList This output is InPlace with the input

Parameters

Max_Step Double The maximum step that the point (2D/3D) could do Filtering value Double The value defines the amount of filtering smoothing. If

the value is greater then one the filter is more reactive Remarks Use a kalman filter based on cinematic model (X' = V + nx , V' = nv) to filter the MOCAP_POINT_2D and the MOCAP_POINT_3D input. Every components of the points are filtered separately. The 'Filtering value' is used only for Kalman filter initialization. The points are recognized by label parameter of MoCap Datatype

2. MoCap.Input.EmptyGenerator, MoCap.Input.EmptyGeneratorSync

CLSID: {4D7F948B-9817-4174-AB40-C82264D99415}

Inputs

SyncIn Any Only in EmptyGeneratorSync Outputs

MoCap List MoCapDataList <Notes> Parameters

Period Integer Period of activation Erase on Activate Boolean If checked the content of the MoCap List is erased

Remarks

Generate an Empty list of MoCap items.

3. MoCap.Item.Add CLSID: {36E5768C-EFC2-4C08-9D88-04D515D9C749}

Inputs


Outputs

MoCap List MoCapDataList The Output is in place

Parameters

Label String Label of new item Type Combo Type of an item:

Scalar Point 2D Point 3D Vector 2D Vector 3D Bound 2D Bound 3D

Parameter 2-7 Double Value of the item.

Factor Double Normalization Factor

Reliability Double Item Reliability

Description String Description of the item

Remarks

The block adds a new item in a MoCap List. The Type Parameter define the number of value are enabled. The following table describe any value: Parameter 2 Parameter 3 Parameter 4 Parameter 5 Parameter 6 Parameter 7 Scalar Scalar Value - - - - - Point 2D X Y - - - - Point 3D X Y Z - - - Vector 2D Dx Dy - - - - Vector 3D Dx Dy Dz - - - Bound 2D X Y Width Height - - Bound 3D X Y Z Width Height Depth

4. MoCap.Item.Extract CLSID: {A4C236D7-07D4-4D7E-BC8E-D330F4C82E47}

Inputs


Outputs

Value Matrix This output Matrix (6x1) holds values of the extracted item (See AddItem to understand these value).

Factor Scalar The item normalization factor

Reliability Scalar The item reliability Label StringDatatype The item label Description StringDatatype The item description

Parameters

Extraction Mode Combo The mode to extract the item. An user can select an item: ByLabel ByIndex

Label String The label of the selected item in the MoCap List. This parameter is enabled only if the ExtractionMode is on ByLabel

Index Integer The index of the selected item in the MoCap List. This parameter is enabled only if the ExtractionMode is on ByIndex

Remarks

This block extracts an item from the input MoCap list and expand it into 5 output.

5. MoCap.Item.Rescale CLSID: {D9983823-9C03-40A8-9CD3-41C5FEABABB8}

Inputs


Outputs


Parameters

Label String The label of the selected item to rescale in the MoCap List.

Factor Double

Remarks

This block rescales the selected item using the inserted factor

6. MoCap.Item.SetNormFactor CLSID: {777BE6B2-EC73-4B2B-AF62-BA3E4F2C0A6D}

Inputs


Outputs


Parameters

Label String The label of the selected item Factor Double

Remarks

This block changes the reliability of the selected item in the MoCap List, but doesn’t rescale the value.

7. MoCap.Item.SetReliability CLSID: {C7D64C68-D012-4911-AC1A-94655337720A}

Inputs


Outputs


Parameters

Label String The label of the selected item Reliability Double Remarks

This block changes the reliability of the selected item in the MoCap List.

8. MoCap.List.AverageReliability CLSID: {8608FBE4-52F6-4E87-942E-8F3292825578}

Inputs


Outputs

Average MathDatatype.Scalar The Average of reliability of the item in the MoCap list

Remarks

The block estimates the mean value of the items in the MoCap List.

9. MoCap.List.NumberOfItems CLSID: {91B879EC-7E28-495B-8CC2-978819FAA518}

Inputs


Outputs

Number_of_items Scalar

Remarks

This block returns the number of items in the input MoCap list

10. MoCap.List.Rescale CLSID: {BEBF3123-7911-4940-BB3A-928FF56CFC4C}

Inputs


Outputs


Parameters

Scale factor Double Remarks

This block rescales all items value of the MoCap List using the inserted factor

11. MoCap.List.SetReliability CLSID: {D6237D7F-78A1-44C6-ACAB-72637D6804C9}

Inputs


Outputs


Parameters

Reliability Double Remarks

This block changes the reliability of items in the MoCap List to the inserted Reliability value

12. MoCap.List.SetScale CLSID: {C7D64C68-D012-4911-AC1A-94655337720A}

Inputs


Outputs


Parameters

Scale factor Double Remarks

This block changes the scale factor of items in the MoCap List but doesn’t change the value of items.

13. MoCap.List.Validate CLSID: {45E654D3-4680-477F-B906-9896D298C012}

Inputs


Outputs


Parameters

Validate Boolean If checked the output list is always valid Remarks

This block changes the state of the datatype valid flag.

14. MoCap.Operations.Extract CLSID: {DD79A92A-4B1F-4EFA-AE56-7951C5F95150}

Inputs


Outputs


Parameters

Extraction Mode Combo The mode to extract the item. An user can select an item: ByLabel ByIndex

Label String The label of the selected item in the MoCap List. This parameter is enabled only if the ExtractionMode is on ByLabel

Index Integer The index of the selected item in the MoCap List. This parameter is enabled only if the ExtractionMode is on ByIndex

Remarks

This block extracts an item from the input MoCap list and generates a new MoCap list that hold the extracted item.

15. MoCap.Operations.Merge CLSID: {1C4EEFC5-94C5-4AD6-837B-5EB56C1A3930}

Inputs

MoCap List MoCapDataList MoCap List MoCapDataList

Outputs


Remarks

This block merges two MoCap List. The second input overwrites the items of the first input with the same label

16. MoCap.Output.Draw2DItem CLSID: {D4477D3B-6262-445E-BEB7-0B6601E553A7}

Inputs


Parameters

DrawPoint Boolean Enabling the points drawing PointColor Command Open the Color DialogBox to select the color of points DrawVectors Boolean Enabling the vectors drawing

VectorColor Command Open the Color DialogBox to select the color of vectors

DrawBrects Boolean Enabling the Bounding Rectangle drawing

BRectColor Command Open the Color DialogBox to select the color of bounding rectangle

DrawLabels Boolean Enabling the Labels drawing

LabelColor Command Open the Color DialogBox to select the color of labels

Remarks

This block draws all the 2D elements in the input MoCapDataList. The drawable Items are 2D points and 2D bounding rectangles. It also draws a 2D vector, but only if it is paired to a 2D point, representing the vector application point. To pair a point to a vector, if the vector label is LABEL, the point label must be %%LABEL

Documents

EyesWeb 3 Legacy Library Quick Reference