Monitor

To view the status and progress of the current action select the “monitor” item in the menu pane. This is the default entry when accessing the Projects part of DATPROF Runtime.

The Monitor is the main screen for running applications.

  • If no application is installed it will show a message indicating the application should get installed.
  • During installation you can monitor the progress of the installation
  • Once installed it shows the name, version and generation date of the application and all the scenarios with a start button.
  • Once running the status of the actions is visible.

The scenarios are listed on the right side of the screen.

Click on the name of the scenario to start it.

At the right side of the scenario button is a on off button indicating to make the button Visible or Hidden in the Dashboard. 

(info)
 Users with only the User role will only be able to see and start the visible scenarios. 

Parameters


A DATPROF generated application may have several Parameters defined which should be available at runtime. They must be defined in the parameters section of the environment. Access the specific environment.

In the Menu pane click “Parameters” from the NAVIGATION group.

Below the mandatory parameters for the DATPROF applications for every environment.

DATPROF Integrate

Name

Description

DELETE_THRESHOLD
Default: 100000

Determines the delete strategy.
< DELETE_THRESHOLD: use delete
=> DELETE_THRESHOLD: use table creation with select statement.

PARALLEL
Default: 8

The number of processes to start in parallel.

QUERY_SLAVES
Default: 1

The Oracle SQL*Plus  setting for parallel DML execution. (Only Oracle)

RUNS_TO_AVERAGE_LOAD
Default: 5

This applies if more processes may execute at the same time as the parallel setting allows. The process with the highest load factor goes first. For processes that have an equal load factor the process with the highest average load takes priority. The value sets the number of runs to calculate the average for.


DATPROF Subset

Name

Description

DPF_TEMP_SCHEMA
Default: From original Subset template

The schema used for the Subset temporary tables.

DPF_MAX_FUNCTIONAL_ITERATIONS
Default: From original Subset template

The Maximum functional iterations.

DPF_AUTO_RETRY
Default: From original Subset template

The number of minutes to wait before on error and retry.

DPF_DATABASE_LINK
Default: From original Subset template

The name of the database link to use (only oracle)

DPF_BYPASS_LOGGING

Default: True

This setting effects the database logging on a table. Default this bypassing is enabled (true).

Normally disabling logging speeds up the Subsetting.


DATPROF Privacy

Name

Description

DPF_AUTO_RETRY
Default: From original Subset template

The number of minutes to wait before on error and retry.

DPF_BYPASS_LOGGING

Default: True

This setting effects the database logging on a table. Default this bypassing is enabled (true).

Normally disabling logging speeds up the anynomization.


Hidden parameters

Besides these mandatory environment parameters there is one hidden parameters that can be used . This one is not visible in the Parameter settings screen but does have a default value. If you provide this parameter in the application parameters screen you can override the default value.

Name

Description

DPF_SCRATCH_DIR
Implicit Default: <Agent Directory>/run/scratch

The temporary directory for this environment. Used for constraint checks.


SQL*plus parameters

When using the SQL*Plus module you can specify several settings that define a SQL*Plus session and pertain to all SQL*Plus modules (modules that are executed in SQL*Plus). Refer to your Oracle documentation for an exact explanation of these session settings. The SQL*Plus properties should get prefixed with ‘sqlplus.’:

Name

Description

sqlplus.current_schema

See Oracle documentation.

sqlplus.cursor_space_for_time

See Oracle documentation.

sqlplus.filesystemio_options

See Oracle documentation.

sqlplus.hash_area_size

See Oracle documentation.

sqlplus.optimizer_features_enable

See Oracle documentation.

sqlplus.optimizer_mode

See Oracle documentation.

sqlplus.session_cached_cursors

See Oracle documentation.

sqlplus.sort_area_size

See Oracle documentation.

sqlplus.statistics_level

See Oracle documentation.

sqlplus.sql_trace

See Oracle documentation.

sqlplus.timed_statistics

See Oracle documentation.

sqlplus.force_parallel

See Oracle documentation.

sqlplus.enable_parallel

See Oracle documentation.


There is one Sqlplus module setting that is special. This is not a SQL*Plus setting but a modifiable Runtime parameter effecting the execution of the sql script. This one is also prefixed with “sqlplus.”.

Name

Description

sqlplus.purge.recyclebin
Implicit Default: yes

 

 

Normally every execution of an SQL*Plus module is preceded by purging the Oracle recycle bin.

You can override this default behavior by setting this parameter to: n, no, off, false. (case insensitive)

Any other value, or if the parameter is not present,  will purge the recycle bin.

In DATPROF Integrate you can use your own parameters. To use these parameters in DATPROF Runtime they must be added to the environment. Enter its Key(name)  and value in the dialog in the left part of the Runtime pane and click “Add parameter”.

Parameters can be updated and deleted

Note: Parameters should be entered for every individual environment. Importing a properties file can be helpful when having a lot of parameters.

For Microsoft SQL Server, Runtime does not provide Sqlcmd parameter settings.

Database

The database menu allows you to modify the database connection details for the current database type. The dialog is a stripped version of the one used to create an environment.

You cannot change the database type. 

(warning)
 Changing database parameters may hinder your application.

Run history


Every run of the applications creates a log file which can be viewed under Run history.

Access the specific environment.


In the Menu pane click “Run history” from the NAVIGATION group.

Every finished run of the application in this environment has a row identified by a unique id-number.

Also , the start date and time, it’s finish date and time and the final status are listed

For a in depth detail view of the run click the link of the ID.

Every action has one record stating its Name, (Module) Type, Status, Start/End date (and time), Duration in milliseconds, (Re)Try count and Cycle number(whenever iterative).

You can filter the view by using the search bar and sort the rows by using the column headers. The default sorting is based on the Start date. 
When using the search bar the value is searched for in all available fields.  
You can adjust the maximum number of entries to view. 50 is the default and 100 is the maximum.

Name:                   This is the name of the action, It’s a link to the output of that specific action.

Type:                     The action type as defined in Integrate or generated by DATPROF Subset and Privacy.

Status:                   An action has the following possible statuses:

  • Done:                            The action has successful finished.
  • Warning:                      The action has finsihed with warnings.
  • Error:                            The action did end with an error.
  • Ignored:                        The action was in error but on user request it was ignored.
  • Skipped:                       The action is skipped.

You can filter the modules on the status by selecting the big buttons mentioning the status and the amount of modules belonging to it.

Included modules are white. Excluded modules are grey. Default you will not see skipped modules.


(warning)
 An action in error that is ignored on user request will show up twice in the history: one time in error and one ignored!

(warning)
 In DATPROF Subset all the SQL for the different scenarios are generated and available in DATPROF Runtime. They show up in the Run History as Skipped modules!


Start date:            The date and time the action started.

End date:              The date and time the action has finished.

Duration:              The elapsed time for the action in milliseconds.

Try:                         The number of times the action is manually set to retry. Normally it is 0.

Cycle:                     Actions which containing a loop will have a cycle count. This number is shown here.

You can view the result of every individual action by clicking on the link of an action. This will show the log message of this action only.

Click “Back” in the Menu pane under NAVIGATION to return to the list of Run history row’s

To reduce the list of historical runs you can purge this list by providing the amount of oldest entries you want to remove. Use the button “Cleanup history…” for a dialog to provide the amount of logging rows to remove.


Run history details

On the right side the Details of the run are listed: Projectname, Environment, application name. Scenrio, Start date, End date, Duration and status.

This is an example of a DATPROF Subset run.


Files

Under the Run Details you can find the Files section,

Files stored in the folder using the Internal parameter DPF_ARTIFACTS_DIR are listed her.

You can use this internal paramater for your own scripts.

DATPROF Runtime uses DPF_ARTIFACTS_DIR to save the restoredDDL.sql .


Run.log

You will also find a formatted summary of the Log history.

This is the file RUN-<runid>.log.  This file will be generated when you click on it.


It shows the timeline of the run and summarizes all the Run details.

Every started or finished module has a seperate row so you can sort and filter on start and finished time.

Every row constains a Date, Timestamp, Begin or Finish Keyword, Module type, Modulename and some additional info.

This additional info is for started actions the Try count and Cycle count.

This additional info is for finished actions the Duration.


Below an example.

Project: Project1
Environment: Env1
Application: Gasware-ora (1.2.3)
Scenario: (Re-)Create
Start date: 2018-04-18 16:10:47.84
End date: 2018-04-18 16:11:45.05
Duration: 00:00:57.208
Connection: jdbc:oracle:thin:TEST/*****@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=testserver)
(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=testservice.testdomain)))
Status: DONE

2018-04-18 16:10:48.72 start Process Gasware-ora (try: 0, cycle: 0)
2018-04-18 16:10:48.82 start Process Subset prepare (try: 0, cycle: 0)
2018-04-18 16:10:49.02 start sqlplus Drop Table DPF_LOG - prepare (try: 0, cycle: 0)
2018-04-18 16:10:49.38 finish sqlplus Drop Table DPF_LOG - prepare (duration: 00:00:00.364)
2018-04-18 16:10:49.52 start sqlplus Create Table DPF_LOG (try: 0, cycle: 0)

<snip>

2018-04-18 16:11:44.00 finish Process Drop temp tables (duration: 00:00:00.101)
2018-04-18 16:11:44.10 finish Process Subset cleanup (duration: 00:00:10.372)
2018-04-18 16:11:44.20 finish Process Gasware-ora (duration: 00:00:55.484)
end-of-file