Introduction

This chapter describes the required activities to run and manage applications from the Envrionment Monitor screen.

Prerequisite for this chapter is the successful upload and Installation of an application in an environment and at least one available agent.

Every running application uses one Agent. Once finished this agent may be reused by other applications.





Run an Application by starting a scenario

To start an application go to the Projects tab and select an environment from one of the available projects. The Monitor option is selected and the Runtime canvas shows the status of the Environment and the Application.

Here you can start, view, pause and abort a run of the application.  

Scenarios

An application consist of multiple modules to get executed. DATPROF Runtime can execute a selection of all the modules in the application.

Such a limited selection of modules is called a scenario.

(info) To start an application you start a scenario.

  • DATPROF Privacy applications only have one scenario: "anonymize"
  • DATPROF Subset applications always have multiple scenarios. The four different Deployment Strategies are implemented as scenarios.
  • DATPROF Integrate application has at least one scenario, "any"  and may have many more scenarios developed  by the DATPROF Integrate developer available

DATPROF Subset scenarios in DATPROF Runtime



(info) Modules not used within the started scenario will be visible as skipped in your logging




Managing modules

A running application will show the running modules one by one and when the runa has finished it displays the History log of the run. 

Every module always has a Status, the following statuses are possible:

  • Running:   The module is in execution
  • Done:        The module has successfully finished
  • Warning:   The module has finished but received warnings        
  • Error:        The module has not finished but encounters an error. User action is required.
  • Skipped:   The module is skipped due to the scenario flow.
  • Ignored:    The module is ignored on users request.
  • Waiting:    The module is waiting for a condition to become true to proceed.
  • Looping:   The status of a process module with a wait condition to check it’s condition. This is DATPROF Integrate specific 



Start, Pause, Abort

To start an application click “Start” for the given Scenario.

The Start button will change into two buttons: Pause and Abort. Other buttons are disabled.


Pause: No new modules are started, running modules will proceed executing. The Abort button is disabled and the Pause button is replaced by a Resume button. To continue click Resume.

 

JKUTODO

Abort: No new modules are started, running modules will proceed executing. The Resume and Abort buttons are temporary replaced by one button Abort until the running modules are finished. When these are finished History of this run is shown. Only modules which ran are shown. the Information pane will show the status Abort.  


(warning)
The scenario has aborted, not the module.  Already started modules will finish.


Once Running, the Runtime canvas will show every Modules when executed. Past moduels are visible in the Run history. Future modules are not visible. The number of Modules visible is determined by the PARALLEL setting in the parameters section. 

 

Navigation

  • You may access any other part of the application during a run of the application. You can later come back and the monitor will show its current state.
  • There is no limit of viewers. Other users may open their browser, login and have a look at the same Monitor. You may also have several browser tabs with the monitor open.
  • To have a look at the modules which have finished you can access the Run history during the run. Click on Run History from the NAVIGATION group in the Menu pane.
  • From the Projects main page you see per environment the status.

Under status a descriptive summery is given and the progress bar is visible.

 






Handling Errors


When Running your application errors may occur. Any module in error will prevent the application from finishing.


You have three option to handle errors:

  1. Investigate the error and decide how to respond:
    Then choose to: 
    • Ignore the module
    • Retry the module
  2. Ignore this module from the monitor. 
  3. Retry this module from the monitor.
    To Ignore or Retry a module directly from the monitor is normally not the best way to handle your errors. You may do when you know your application or environment very well.



To investigate the error click on the link of the module name.

The Runtime Canvas shows an overview of the Message and the error.

  • The Left hand side shows the navigation for this action in Error: 
    • Logging, 
    • Code, 
    • Action Parameters)
  • The Centre part shows the logging of the error.
  • The right hand side shows the meta information of the action.

It opens with the Logging action opened. 

Logging 

The logging shows the output of the script and the result from the database.

This where you try to figure out what causes the error.

Some errors are related to issues outside the application. Think of Database problems, permissions or issues on your file system.

Other issues are code related. The can be fixed in the code section of the error.

Other issues are due to user input errors DATPROF Privacy or Subset. Thet can be fixed in the Action Parameters section of this error.

Code

The code shows the script prior to parsing it using Velocity prior to executing it.

The code you see is editable and you can make modifications. Normally only user code should be edited. The code of Templates should not be edited. 

Afther modification you have two options

  • Apply the new code for this Run only. 
    Every new run will give the same error.
  • Apply the new code for this run and also the future runs of this application in this environment. 
    Every new run will use the modified code. 
    This will mark the environment as Patched in the project details in the monitor.

In both cases you create a discrepancy between the source of the generated code and the current runtime code.

After modifying the code in Runtime you should make these modifications also in DATPROF Integrate, Subset or Privacy.

The reason for having this option is to speed up development.

Code editing applies to 

DATPROF Privacy

  • SQL Scripts 
  • OS Scripts

DATPROF Subset

  • SQL scripts
  • OS Scripts

DATPROF Integrate

  • Extract, when a parameter files is used the parameter file.
  • Excel loader, the mapping file
  • Advanced mapping, the sql code.
  • Constraint check, the sql code.

Oracle:

    • SqlPlus, the sqlplus code.
    • SqlLoader, the control file.

Microsoft SQL Server:

    • Sqlcmd, the sqlcmd code.
    • BCP, the format file.

Action Parameters

DATPROF Privacy and Subset use templates for the anonymization and subsetting actions. These templates have a generic setup and can be reused for identical actions. Specific parameters are injected for the specific situation. These parameters are the Action Parameters.

All user input in DATPROF Privacy and Subset is provided as Action Parameters and can be edited in Runtime when an error occurs.

An example is the condition in DATPROF Privacy or the Startfilter in DATPROF Subset.

Internal parameters are not editable.

DATPROF Integrate does not have Action Parameters.

Example 1: Error in user code

The first one is from a DATPROF Integrate project. It handles a coding issue. For DATPROF Privacy or DATPROF Subset SQL/OS Scripts this works the same.


Error Example 1

The Information pane shows some metadata of the Action.

The Runtime canvas shows the errors in detail. In this case we have an ORA-00904: "EE": invalid identifier 

This is an application error caused by invalid user code in the DATPROF Integrate project. 

(info) When having an error in the database (permissions, full logfiles etc)  your modification is in the database and not in your code.


In this case , to solve the error select Code from the menu. The code of the script now becomes editable.

The script you then look at is the SQL code executed prior to the parsing using Velocity prior to executing it.

In this module only plain SQL code is used. Changing "EE" into 'EE' will fix the error.




Example 2: Error in generated code

The second example shows an error in generated code by DATPROF Privacy.

DATPROF Privacy and Subset generated code results in generic Velocity Template code. When running Action Parameters are injected to perform the specific action.

Errors in generated code may confuse you when looking at the logging and the code.

The Logging shows the output of the code and helps you to investigate the error , likewise scripts.

The Code  shows the velocity template code and does not look like SQL code at all. You can have a look at it but do not change it!.


Logging

It shows that TYPES is and invalid identifier. This is used in the insert ALL when expression. This is a column name, it should be TYPE.

This is condition on the Blank function set in DATPROF Privacy.


Code

Now check the code, this is quite different then the SQL script code. 

 


Action Parameters

Generated code uses Action paramaters, some of these are entered by the user in DATPROF Privacy or Subset.

User defined settings can be modified when encountering errors.

In this case the condition is invalid. It should be: type is null

Action parameters can only be saved for the current run.