Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 1
EDAMIS Web Application
user guide
for data sender
Project EDAMIS
Author Data and metadata services and standards, Unit B5
Release 1.1
Status Final
Reference QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 2
TABLE OF CONTENTS
1 INTRODUCTION ......................................................................................................................................... 3
1.1 PRINCIPLES ................................................................................................................................................ 3 1.2 USER INTERFACE ....................................................................................................................................... 3 1.3 THE EDAMIS MESSAGES ......................................................................................................................... 4
2 WHERE TO START FOR SENDING DATA TO EUROSTAT USING THE EWA? ........................... 4
2.1 GET YOUR USER ACCOUNT......................................................................................................................... 4 2.2 LOGGING IN FOR THE FIRST TIME ............................................................................................................... 4 2.3 MAINTAINING THE USER ACCOUNT ............................................................................................................ 6 2.4 EUROSTAT CONNECTION STATUS ............................................................................................................... 6 2.5 LOGGING ON AND LOGGING OFF ................................................................................................................ 6
2.5.1 Logging on ...................................................................................................................................... 6 2.5.2 Automatic Logging off ..................................................................................................................... 6 2.5.3 Logging off ...................................................................................................................................... 6
3 SEND DATA .................................................................................................................................................. 7
3.1 INTERACTIVE MODE ................................................................................................................................... 7 3.2 SEMI AUTOMATED MODE ........................................................................................................................... 9 3.3 FULL AUTOMATED MODE ......................................................................................................................... 10
4 MANAGE SAVED DATA FILES ............................................................................................................. 12
5 RECEIVE DATA FILES ............................................................................................................................ 13
6 ACTION HISTORY ................................................................................................................................... 14
7 GLOSSARY ................................................................................................................................................. 15
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 3
1 Introduction
The EDAMIS Web Application (EWA) is a web based application which allows a user to
send data files to Eurostat. This user guide explains new users how to get started with the
application. Some basic but important issues are addressed in this document beginning from
the registration and the statistical data transmission to the monitoring of the data files traffic.
1.1 Principles
EDAMIS is a complete environment that offers data transmission and management services in
the European Statistical System. It stands for "electronic Data files Administration and
Management Information System". It includes:
1. Advanced data transmission facilities for all Eurostat data providers, mainly based on
three main approaches: the EWA (EDAMIS Web Application), the EWP (EDAMIS Web
Portal) direct upload system and the EWF (EDAMIS Web Form) which looks like a
spreadsheet.
2. Advanced data reception features at the Eurostat "Single Entry Point" (SEP), which is the
EDAMIS server
3. A reporting system that provides reports on traffic
The EWA is a local HTTP server that enables the secured transmission of data from Member
States to Eurostat. The statistical data that needs to be transmitted to Eurostat is made
available to the EWA, which converts it into a standardised message and uses a
telecommunication component (STATEL – hidden for the data sender) to transfer the
message to the EDAMIS server in Eurostat. Based on the metadata associated with the data in
the message, the server knows where and how to deliver the data. The server then sends an
acknowledgement to the EWA to inform the user that the data has been delivered.
1.2 User Interface
The EWA is used through a Web browser such as Internet Explorer or Firefox, with the EWA
server performing all the functions, such as sending data to the EDAMIS server, receiving and
processing acknowledgements, or encrypting confidential datasets. No applets are
downloaded, and the browser remains the visible interface of EWA. As a result, no
certificates must be installed or accepted, and the operation of EWA is easy to master.
EDAMIS Help Centre and Support
Note A set of comprehensive documents are available at the EDAMIS Help Centre. They cover a wide
range of topics like the available software, including very technical aspects for some of them and
the advices to select the best method to fit your requirements as well as the installation
procedures. You can look at the EDAMIS Help Centre in the following link:
https://webgate.ec.europa.eu/edamis/helpcenter/website/index.htm
The support team is at your disposal for any questions, incidents resolutions or you want to report
or to resolve any problem encountered when using the data collection tools. Do not hesitate to
contact the support team for any problem or question related to data collection.
Mail to: [email protected]
Tel: (+352) 430133213
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 4
1.3 The EDAMIS Messages
The EDAMIS system is based on sending and receiving messages. Unlike many message
based systems, there is no size limit to EDAMIS messages, so sending or receiving a dataset
of several hundreds of megabytes is just as feasible as sending a 10 kilobyte dataset. Previous
implementations used the EDIFACT standard; this implementation uses XML formatted
messages. The logical contents of these messages are very similar, but the additional
functions offered by EWA require additional information in the messages.
2 Where to start for sending data to Eurostat using the EWA?
2.1 Get your user account
Obtain the URL from the administrator of the local EWA, who usually is your “Transmission
Coordinator” or “the Local Coordinator” in your country. You will find an up-to-date list in
the following link https://webgate.ec.europa.eu/edamis/helpcenter/website/overview/lc.htm
Your administrator will provide with a user-id, which in principle, should identical to your
ECAS user-id, if you have one.
2.2 Logging in for the first time
Once you have both, the URL and your user-id, you can open your web browser and copy the
provided URL in the address filed. This URL is the internal hyperlink under which the EWA
has been installed and it looks like , or it can also be an address –
(depending on your network configuration) -- like http://192.168.1.100/Login as it appears on
the login mask below (See fig 1)
Fig 1: EWA login mask
When logging in for the first time, you must only introduce your User-ID and leave the
Password empty. The registration process will be completed at the first login, where you will
have the possibility to specify your own password (See fig 2).
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 5
Fig 2: User Management screen
In the user management window, the data sender will need to complete the form by
introducing a password which is mandatory as indicated by the asterisk nearby. The data
sender will have to confirm the password (which is hidden) and is invited to complete the rest
of the form if necessary. Once this is done, the data sender will click on the [Update] button to
save the changes (See Fig 3).
Fig 3: User Management details
Lines on short activity report field
Note When the data sender logs in the EWA, a short activity report is displayed. The data sender has
the possibility to specify the number of lines which will be shown by entering the corresponding
number in this field. If left to zero, no short activity report will be displayed.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 6
2.3 Maintaining the user account
Once the registration is completed, the full main menu is made available and a confirmation
that the user management parameters have been updated (See Fig 4).
Fig 4: EWA main menu
Later on, if the data sender wants to update the user data, this can be done via the menu
“ ”. This will bring the user to the “User Management” screen as in fig 2.
2.4 Eurostat connection status
The upper right corner of the EWA window shows the status of the connection to Eurostat. If
the status line is green (See fig. 5), datasets will be transmitted to Eurostat. If the status line is
red, there might be a communications problem and it would be advisable to contact your
EWA system administrator or the Local Coordinator.
Fig 5: EWA connection status
2.5 Logging on and logging off
2.5.1 Logging on
The data sender logs in with its User-ID and password when the logging mask is displayed.
2.5.2 Automatic Logging off
After a configurable period of inactivity (3 hours by default), users are automatically logged
off. It is however recommended to use the “Log off” main menu
http://app2.EDAMIS.net/clientdocs/logoff.htmlentry to terminate the connection with the
EWA once the work has been completed.
2.5.3 Logging off
The data sender can press the “Log off” main menu entry to log off from EWA.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 7
3 Send data
3.1 Interactive mode
Sending a data file to Eurostat in an interactive mode is as simple as this:
1. Log on to the EDAMIS Web Application
2. Select “Send Data file”
3. Browse for the file to transmit
4. Complete the metadata as requested
5. Click Perform Transfers
By using the main menu “Send Data File”, the “Send Data File” screen appears (See Fig 6).
Fig 6: Send Data File screen
The screen areas are as described in the table below:
1 File name
This is the name of the file which will be sent to Eurostat. It can be whatever file, file
type, file size. EDAMIS Web Application will automatically complete the “send data
file” input form, provided that the file name is structured according to the dataset naming
convention or that the file is GESMES formatted. This concerns the dataset, the year, the
period and the action fields
2 Dataset
In the dataset drop down box, there is the list of datasets that the user is supposed to send
to Eurostat as configured by the administrator of the EDAMIS Web Application.
3 Year
This is the year of the reporting period, for example “2008”.
4 Period
This is the reporting period and it depends on the periodicity of the dataset. This field is
greyed once it is not relevant. For instance, it is not visible for yearly dataset since the
year itself is sufficient to determinate the coverage of the reported data.
5 Action
The action code is only given as information and has no effect on the data transmission
chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”. When used, there
must be an agreement about the meaning of the action code between the production unit
at Eurostat and the data providers in Member States.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 8
6 Encrypt?
In the EDAMIS inventory, when a domain has an encryption key attached to it, the
encrypt flag is automatically proposed to the data sender. When ticked, the data file
which is transmitted will be encrypted with the public encryption key attached to the
domain whose the dataset belongs to. At that time, only the owner of the corresponding
private key (domain manager at Eurostat) will be able to decode the ciphered data file.
7 Valid?
In the EDAMIS inventory, when a dataset has defined validation rules, the valid flag is
automatically proposed to the data sender. It will then be possible to check the coherence
of the data file against the defined validation rules before sending the data to Eurostat.
The [Add/Validate] button will launch the validation and the validation report will
appear in a separate window. If the validation does not report validation errors, then the
[Perform Transfer] button will be enabled allowing the data file to be transmitted. If the
validation fails, then the data sender must correct the corresponding file and reprocess it
again. In any case, the data sender is able to “force” a transmission using the [Force]
button even if the data file does not match with the validation rules.
8 Force
[Force] button to be used to “force” a data file transmission
9 Clear
The [Clear] button allows deleting a row from the “send data file” input screen.
10 Free text comments
This is a place where the data sender can leave a free text message at the attention of its
counterpart at Eurostat. It has a length of 256 characters.
11 File with explanatory notes
When there is a need to provide the used methodology, the explanatory note is used. This
allows the data sender to attach a file which has a maximum size of 2Mb. It will be
delivered to the statistician in charge of the dataset together with the notification
message.
12 Browse
The [Browse] button allows the data sender to “browse” the directories in which the data
files are located.
13 Add/Validate
The [Add/Validate] button has two functions. It allows building up a list of several data
file to be transferred to Eurostat, in case of multiple transmissions and if validation rules
exist for the dataset(s), it performs the validation of the attached data file.
14 Perform transfer
The [Perform transfer] button, used at the final stage, really performs the transmission of
the data file(s) which has/have been selected with the [Add/Validate] button.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 9
3.2 Semi automated mode
The semi automated mode allows a third party application to deliver data files in a particular
directory of the EWA belonging to a specific user. Usually, EWA is installed by default under
the \\EWA directory on a dedicated server. Under the “EWA” directory there is a “user”
subdirectory containing all defined EWA users. Therefore, all the EWA users have their own
user directory where the data files which need to be sent to Eurostat can be placed (See fig 7).
Fig 7: the “user/sender” directory
In the above-example, the EWA application knows three users which are the “Administrator”
and the users “sender” and “u1”.
Once the data files have been dropped in the user directory, for example in the user directory
of the user “sender”, next time this user will log into the EWA, and will go to the “Send Data
file” main menu, a list with all the data files which were dropped in the “sender” directory,
will appear pre-filled in the screen in a way that the data sender (here called “sender”) can see
all of them. The user will then just need to confirm (or not) that data files in the list must be
transferred (or not) by clicking the [Force] button (See fig 8).
Fig 8: The pre-filled “Send Data File” screen of the user “sender”
In this way, the data sender will have more a role of controlling what must be sent since EWA
will have prepare the job.
The third party application which drops the data files in the appropriate directory could be a
robot, a third party application or a data warehouse, but it can simply be done via a windows
browser using the drag and drop technique.
Access rights to the directory \\EWA\user\sender
Note The network administrator must allows the third party application to have write access right to
this directory otherwise the application will not be able to drop files in the special EWA directory
and therefore the semi-automated mode will not be enabled.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 10
3.3 Full automated mode
The full automated mode allows a third party application to deliver data files in a particular
directory of the EWA belonging to a specific user. Usually, EWA is installed by default under
the \\EWA directory on a dedicated server. Under the “EWA” directory there is a “user”
subdirectory containing all defined EWA users. In each user sub-directory, there is an “EDI”
sub-directory. Once data files are placed in this specific directory, EWA considers that these
data files must be automatically be sent to Eurostat (See fig 9).
Fig 9: the “user/sender/EDI” directory
EWA is regularly browsing all EDI users’ sub-directories and whenever a data file is found,
EWA initiates an automatic transmission to Eurostat.
The mechanisms which enable the full automated mode rely on the use of the dataset naming
convention for building the data file name (which is the envelope) or on the use of the
GESMES formatted data files.
Whenever a data file cannot be processed automatically due to some inconsistencies, EWA
will automatically switch to the semi-automated mode, allowing the data sender to complete
the missing information before performing the transmission.
More information about the dataset naming convention are found in the following link
http://circa.europa.eu/irc/dsis/EDAMIS/info/data/website/practice/DSNC.htm (See fig 10)
Fig 10: Extract of the dataset naming convention
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 11
Access rights to the directory \\EWA\user\sender
Note The network administrator must allows the third party application to have write access right to
this directory otherwise the application will not be able to drop files in the special EWA directory
and therefore the semi-automated mode will not be enabled.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 12
4 Manage Saved Data files
By using the “Send Data file >> Saved Data files” menu, the data sender is able to manage the
backup copies of the data files that have been sent to Eurostat (See fig 11).
Fig 11: Manage Saved Data File screen
The screen areas are as described in the table below:
1 Date
This is the date of the sent data.
2 User
This is the user who sent the data.
3 Data file name / notes name
This is the name of the file which will be sent to Eurostat.
4 Domain
This is the name of the domain to which belong the dataset.
5 Dataset
This is the name of the dataset.
6 Year
This is the year of the reporting period, for example “2008”.
7 Period
This is the reporting period.
8 Action
The action code is only given as information and has no effect on the data transmission
chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”.
9 Size
This is the file size of the data file sent.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 13
5 Receive Data files
When the EWA is used to receive datasets, the “Receive Data File” menu entry displays a
form with a list of the datasets that have been received. Click the appropriate dataset to
download it to your PC (See fig 12).
Fig 12: Download Received Data Files screen
The screen areas are as described in the table below:
1 Date
This is the date of the sent data.
2 Data file name
This is the name of the file which will be sent to Eurostat.
3 Domain
This is the name of the domain to which belong the dataset.
4 Dataset
This is the name of the dataset.
5 Year
This is the year of the reporting period, for example “2008”.
6 Period
This is the reporting period.
7 Action
The action code is only given as information and has no effect on the data transmission
chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”.
8 Size
This is the file size of the data file sent.
9 Free text
This is the message which has been given by the data sender.
10 Explanatory notes
This is the link to the file which was attached as explanatory notes for the sent data file.
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 14
6 Action History
The “Action History” main menu entry gives access to the following commands (See fig. 13):
Fig 13: Action history menu
1. Short: an on-screen report restricted to the default number of lines set by the
Administrator or in the user account (see Maintain Account).
2. Long: produces an on-screen report of all the actions
3. Month: an on-screen report restricted to the entries for the current month
4. CSV: Send: generates a CSV file with all the Send actions
5. CSV: Receive: generates a CSV file with all the receive actions
6. CSV:Admin: generates a CSV file with all the administrative actions
QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 15
7 Glossary
Concept Definition
CSV Comma-Separated Values file format.
Data file Physical data file transmitted which should normally correspond
to an instance of a dataset (a physical instance of a dataset
occurrence).
Dataset Dataset structure with a specific periodicity and to which is
usually linked a deadline for transmission. The data content of a
dataset is homogeneous.
Dataset naming
convention
Convention used by all production units at Eurostat for uniformly
naming a dataset. It composed of the domain name, the statistical
table name and the periodicity, all separated by an underscore.
The dataset naming convention is recognised by all third-party
applications at Eurostat.
Dataset occurrence Occurrence of a dataset for one country and one period or time
series or sequence. For Web Forms it is called a “Web Form
instance”.
Dataset structure Concept of one single data structure associated to one or several
statistical tables (closely linked into the same statistical domain).
Domain Consistent group of datasets that are closely linked together
(belong to the same statistical area and based on the same legal
act(s) or voluntary agreements).
EDAMIS electronic Data files Administration and Management Information
System.
EWA EDAMIS Web Application.
EWP EDAMIS Web Portal.
SDMX-ML XML format for the exchange of SDMX-structured data and
metadata. The SDMX format (Statistical Data and Metadata
Exchange) is the result of cooperation between the BIS, the ECB,
Eurostat, the IMF, the OECD and the UN to explore common
e-standards and ongoing standardisation activities that could
afford more efficiency in the field of statistical information.
Table Statistical table as defined in a legal act or a voluntary agreement
(with records made of fields).
Theme Theme of the Statistical Programme.