Main

SLA Engine Component 2.0 (minor-edit)

2017-09-06T15:51:34Z

1 Summary
2 Component responsible
3 Architecture design
4 Installation manual
- 4.1 Development
- 4.2 Production
5 User manual
- 5.1 API Specification
6 CloudSocket extensions to WS-Agreement

Summary

The SLA Engine represents the component responsible for generating, storing and observing the formal documents describing electronic agreements between the parties involved in CloudSocket: BPaaS bundle customers, brokers and cloud providers. The component follows the WS-Agreement specification. This means that the documents representing templates and agreements are valid according to the schema defined in the specification. The specification is extended where needed to cover some specific CloudSocket needs.

This component will expose:

A graphical user interface for checking the status of agreements,
A REST API interface allowing programmatic access to the different types of functionalities offered.

The main functionalities are:

Generation of WS-Agreement templates and agreements.
Management of SLA related entities: templates, agreements, providers, violations, penalties
Assessment of SLOs and corresponding penalties when an SLO is violated.
Notification of detected violations and incurred penalties to interested parties.
...

Type of ownership	Extension & Adaptation
Original tool	SLA Framework asset
Planned OS license	Apache License Version 2.0.
Reference community	None

Consist of

SLA Engine (core functionalities)
SLA Dashboard (Graphical User Interface)
SLA Generator (generates WS-Agreement compliant documents)

Depends on (uses or used-by dependencies)

Monitoring Engine
Allocation Environment
Marketplace
Cloud Provider Engine

Component responsible

Developer	Email	Company
Roman Sosa Gonzalez	roman.sosa@atos.net	Atos
Joaquin Iranzo	Joaquin.iranzo@atos.net	Atos

Architecture design

Figure: General architecture for SLA Engine

The SLA Engine in CloudSocket is composed of three main components:

SLA Core. It relies on the Atos SLA Framework asset, with some CloudSocket-related modules. It contains all the basic functionality. The main subcomponents are:
- Repository. Database of WSAG related entities.
- Assessment. It is in charge of the evaluation process of the SLA agreements. It notifies raised violations and penalties to interested observers (i.e., an Accounting component).
- REST Service. It is the REST interface of the SLA Core to the world.
- WSAG Generator. It generates WSAG-compliant templates and agreements according to the requirements of allocation and marketplace, respectively.
- Colosseum Adapter. It is charge of: i) subscribing into Colosseum to the appropriate metric events, and ii) to receive events and translate them into the internal SLA representation.
SLA Dashboard. This component is a frontend application where the user/admin can check the status of agreements and the penalties that have been applied.

The related components with the SLA Engine in CloudSocket are:

Monitoring. It reports the metric data to the SLA Core.
Allocation. The Allocation Environment contacts the SLA Generator to obtain a template that describes the service level of a BPaaS Bundle.
Marketplace. It shows a human-readable description of a BPaaS Bundle SLA to the customers.
Cloud Provider Engine. It pProvides the SLA to be stored and managed by the SLA Engine.

Installation manual

Development

Requirements

Oracle JDK >= 1.7
Database to install the database schema for the service: Mysql>=5.0
Maven >= 3.2
Python >=2.7

Installation

Download source code from gitlab

 git clone https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/sla-framework.git cd sla-framework

Create database in mysql

 $ mysql -p -u root mysql> CREATE DATABASE atossla; mysql> CREATE USER atossla@localhost IDENTIFIED BY '_atossla_'; mysql> GRANT ALL PRIVILEGES ON atossla.* TO atossla@localhost;

Compile SLA Engine
```
 ( cd sla-core && mvn clean install ) 
```
Execute sla-core server. The following command will start an SLA Engine server in port 8080.
```
 sla-core/bin/runserver.sh [port] 
```
Execute sla-dashboard server. The following commando will start an SLA Dashboard server in port 8000
```
 sla-dashboard/bin/runserver.sh 
```

Production

Requirements

Oracle JRE >= 1.7
Database to install the database schema for the service: Mysql>=5.0
Python 2.7
Virtualenv

Installation

Create the database like done in Development section.

The script `dist/bin/make-dist.sh` creates an archive file with the necessary scripts and artifacts to run the SLA Engine in a server.

 dist/bin/make-dist.sh scp dist/target/sla-framework.tar.gz server: # replace 'server' with the ip/name of server ssh server: tar xzf sla-framework.tar.gz cd sla-framework virtualenv ~/virtualenvs/sla-dashboard ( . ~/virtualenvs/sla-dashboard/bin/activate && cd lib/sla-dashboard && pip install -r requirements.txt && ./manage.py migrate ) bin/start-sla-core-service.sh bin/start-sla-generator-service.sh bin/start-sla-dashboard-service.sh

To stop the services:

 bin/stop-sla-core-service.sh bin/stop-sla-generator-service.sh bin/stop-sla-dashboard-service.sh

Environment variables

The SLA Engine execution can be configured with the use of environment variables. The list of recognized env vars and their default values is:

SLA_CORE_HOST (127.0.0.1), SLA_CORE_PORT (8080): sla-core listening address; passed to sla-dasboard.
SLA_DASHBOARD_HOST (127.0.0.1). public hostname of SLA Dashboard; it must be reached by IDM component.
SLA_DASHBOARD_PORT (8000): sla-dashboard listening port.
SLA_GENERATOR_HOST (127.0.0.1), SLA_GENERATOR_PORT: sla-generator listening address.
DB_URL (jdbc:mysql://localhost:3306/atossla): A jdbc connection string in the format jdbc:mysql://${IP}:${PORT}/${DB}.
CLOUDIATOR_URL (http://127.0.0.1:9000/api): URL of Cloudiator REST API.
SLA_URL (http://127.0.0.1:8080/api): Public address of SLA Engine REST API; used in subscriptions to Monitoring Engine in order to receive metric notifications.
IDM_CLIENT_ID (""): SLA Engine Client ID for OAuth2 authorization flow.
IDM_CLIENT_SECRET (""): SLA Engine secret for OAuth2 authorization flow.
IDM_AUTHORIZE_URL (http://csmarket.ymens.com:9090/oauth/authorize): Authorization URL in OAuth2 authorization flow.
IDM_TOKEN_URL (http://csmarket.ymens.com:9090/oauth/token): Token URL in OAuth2 authorization flow.

User manual

API Specification

The generic API is in SLA Framework API.

The lifecycle of the SLA in CloudSocket is shown in the diagram below.

Figure: SLA Sequence Diagram

Specific CloudSocket API is explained below:

Generate a template

 POST /api/templates

This endpoint takes a simplified JSON definition of a WS-Agreement template and returns the WS-Agreement template, which can be fed into the SLA Core.

An input example: JSON Input for Christmas bundle

The penalties element are described in the SLA Core developer guide (search for Business Rules).

Example:

 curl -X POST http://134.60.64.232:8001/api/templates -d@./cs-wsag-generator-service/src/test/resources/json_christmas.json -H"Content-type:application/json"

The input file is:

 { "templateName" : "Sending Christmas Greetings (with SLA) SLA Template", "bpaasBundle" : "Sending Christmas Greetings (with SLA)", "broker" : "bwcon", "assessment" : "bwcon", "monitoring" : "bwcon", "guaranteeTerms" : [ { "name" : "KPI Hourly CPU_GuaranteeTerm", "sloDescription" : "Hourly Usage of CPU should be less than 80%", "penaltiesDescription" : "3 penalties occurs in one day => 1â‚¬Â discount on the current month", "slo" : { "condition" : "KPIHourlyCPU", "context" : "MeanCPU", "comparison" : "LE", "threshold" : 80.0 }, "penalties" : [ { "count" : 3, "interval" : "P1D", "penalty" : { "type" : "discount", "expression" : "1.00", "unit" : "euros", "validity" : "P1M" } } ] } ] }

The output is:

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <wsag:Template xmlns:cs="http://wsag.sla.cloudsocket.eu" xmlns:sla="http://sla.atos.eu" xmlns:wsag="http://www.ggf.org/namespaces/ws-agreement" wsag:TemplateId="3b4d79ba-307f-40aa-a139-b925efe1149d"> <wsag:Name>Sending Christmas Greetings (with SLA) SLA Template</wsag:Name> <wsag:Context> <wsag:AgreementResponder>bwcon</wsag:AgreementResponder> <wsag:ServiceProvider>AgreementResponder</wsag:ServiceProvider> <sla:Service>Sending Christmas Greetings (with SLA)</sla:Service> <cs:context> <cs:assessment>bwcon</cs:assessment> <cs:monitoring>bwcon</cs:monitoring> </cs:context> </wsag:Context> <wsag:Terms> <wsag:All> <wsag:GuaranteeTerm wsag:Name="KPI Hourly CPU_GuaranteeTerm"> <wsag:ServiceLevelObjective> <wsag:KPITarget> <wsag:KPIName>KPI Hourly CPU_GuaranteeTerm_kpi</wsag:KPIName> <wsag:CustomServiceLevel> <cs:slo> <cs:constraint>MeanCPU LE 80.000000</cs:constraint> <cs:conditionConstraint>KPIHourlyCPU NOT EXISTS</cs:conditionConstraint> <cs:description>Hourly Usage of CPU should be less than 80%</cs:description> </cs:slo> </wsag:CustomServiceLevel> </wsag:KPITarget> </wsag:ServiceLevelObjective> <wsag:BusinessValueList> <wsag:CustomBusinessValue count="3" duration="P1D"> <sla:Penalty type="discount" expression="1.00" unit="euros" validity="P1M"/> <sla:description>3 penalties occurs in one day => 10€ discount on the current month</sla:description> </wsag:CustomBusinessValue> </wsag:BusinessValueList> </wsag:GuaranteeTerm> </wsag:All> </wsag:Terms> </wsag:Template>

The CloudSocket sequence diagram is depicted in figure.

Generate an agreement

 POST /cloudsocket/api/agreements

This endpoint generates an agreement from a template and the needed information context. It is intended to be called when a bundle has been deployed. Both provider (if not yet), template (if not yet) and agreement are stored in SLA Core.

It receives the template in the body, and the context information (currently, only the customer ID) in headers.

Headers:

customer ID: X-CloudSocket-Customer

Example:

 curl -X POST -d@sla-service/src/test/resources/samples/template02.xml -H"X-CloudSocket-Customer: customer-a" http://134.60.64.232:8080/cloudsocket/api/agreements -H"Content-type:application/xml"

Input body: Template

X-CloudSocket-Customer header: customer-a

The output is:

 4679e474-2322-4da2-be75-a0004c7f8df8

Start evaluation

 PUT /cloudsocket/api/enforcements/{agreementId}

This endpoint makes the appropriate subscriptions to the Monitoring Engine and starts the evaluation of the agreement. It is intended to be called just after the CreateAgreement endpoint.

The body contains a CsEnforcementJob. A serialized example is found in https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/sla-framework/blob/master/sla-core/sla-service/src/test/resources/samples/cs-enforcementjob2.json

It returns the enforcement entity in /api/enforcements/{agreementid} in the body.

Example:

 curl -X PUT -d@sla-service/src/test/resources/samples/cs-enforcementjob2.json http://localhost:8080/cloudsocket/api/enforcements/agreement-id -H"Content-type:application/json" -H"Accept:application/json"

Input body:

 { "agreementId" : "agreement-id", "enabled" : true, "composedMonitorIds" : [ 1, 2, 3 ] }

Output:

 {"agreementId":"agreement-id","enabled":true}

CloudSocket extensions to WS-Agreement

There is a number of extensions that have been made to the WS-Agreement specification in order to obtain the CloudSocket objectives:

Context

It is desirable to explain in an SLA which are the entities that are to be involved under which context and role. In the case of the broker acting as a service provider, the broker may outsource some tasks to other entities (e.g., monitoring, execution, evaluation) so supporting parties can also be involved in the SLA.

Taking advantage of the xs:any child of wsag:Context element, a cs:context element has been defined to support the declaration of additional entities in the SLA: assessment and monitoring, at this moment.

CustomServiceLevel

An SLO should include all appropriate information that can assist in its proper monitoring, at least:

definition of the metric
schedule
measurement window
object being monitored
evaluation frequency

In the case of CloudSocket, the metric and SLO are proper and completely defined in the CAMEL document. The SLOs in the SLA document reference the CAMEL entities.

The figure below is a simplified class diagram of the implied CAMEL entities in the description of SLO in the SLA document.

The Monitoring Engine allows the SLA Engine to subscribe to calculations of a MetricContext. Then, the constraint is:

constraint : metric_context operator threshold metric_context : <metric_context_id> operator : LT | LE | EQ | GE | GT threshold : <double>

A cs:slo element describe the CloudSocket SLOs.

Besides the constraint, a cs:slo may contain a human description (shown in marketplace) of the SLO and a window description.

This is an example of an SLO:

 <wsag:GuaranteeTerm wsag:Name="HourlyResponseTimeLessEqual20msConditionGuaranteeTerm"> <wsag:ServiceLevelObjective> <wsag:KPITarget> <wsag:KPIName>ResponseTimeLessEqual20msConditionGuaranteeTerm_kpi</wsag:KPIName> <wsag:CustomServiceLevel> <cs:slo> <cs:constraint>ResponseTimeCompositeMetricContext LE 20.000000</cs:constraint> <cs:description>the mean of response time in 1 hours should be always less or equal than 20ms</cs:description> </cs:slo> </wsag:CustomServiceLevel> </wsag:KPITarget> </wsag:ServiceLevelObjective> <wsag:BusinessValueList>...</wsag:BusinessValueList> </wsag:GuaranteeTerm>

Business values

Penalties apply when an SLO is not fulfilled. SLA Service allows to define several penalties according to the number of occurrences of the violation in an interval of time, defined by the count and interval properties: if "count" violations occur in "interval" time, the penalty applies.

The SLA Engine defined a generic way to define a penalty. A penalty is composed of the following properties:

Type (string): it is the type of the penalty. For example: "discount", "support".
Expression(string)
Unit (string): together with 'expression' represent the value of the penalty. F.e. (100, €) or ("telephone", null) if type="support".
Validity (xs:duration): the time the penalty applies. F.e., if there is a discount of (10, %) and validity is 1 month, the discount is applied over the monthly price. Assume we have a penalty "free_telephone_support"; the validity will be the time this free support will be available.

The penalties are described in the sla:penalty elements, which are inside a wsag:CustomBusinessValueElement.

An example is below:

 <wsag:GuaranteeTerm wsag:Name="HourlyResponseTimeLessEqual20msConditionGuaranteeTerm"> <wsag:ServiceLevelObjective>...</wsag:ServiceLevelObjective> <wsag:BusinessValueList> <wsag:CustomBusinessValue count="1" duration="P1D"> <sla:Penalty type="discount" expression="10.00" unit="eur" validity="P1M"/> <sla:description>if 1 violation occurs in 1 day => apply a discount of 10euro for the current month</sla:description> </wsag:CustomBusinessValue> </wsag:BusinessValueList> </wsag:GuaranteeTerm>

The definition of penalties is not responsibility of the SLA Engine.

The SLA Engine do not handle penalties internally. It detects when they should be raised and communicates them to observers.

Example

A full example of an SLA agreement with CloudSocket extensions follows:

 <wsag:Agreement wsag:AgreementId="b705c5c9-bafa-4f8b-8079-09be96173995" xmlns:wsag="http://www.ggf.org/namespaces/ws-agreement" xmlns:sla="http://sla.atos.eu" xmlns:cs="http://wsag.sla.cloudsocket.eu"> <wsag:Name>Send Invoice IaaS Europe SLA Template</wsag:Name> <wsag:Context> <wsag:AgreementInitiator>84a5e594-f4ce-4daa-ac1e-3ebe032ca97c</wsag:AgreementInitiator> <wsag:AgreementResponder>bwcon</wsag:AgreementResponder> <wsag:ServiceProvider>AgreementResponder</wsag:ServiceProvider> <wsag:TemplateId>94bafed5-335f-4ce3-98f5-e0c57b844997</wsag:TemplateId> <sla:Service>Send Invoice IaaS Europe</sla:Service> <cs:context> <cs:assessment>bwcon</cs:assessment> <cs:monitoring>bwcon</cs:monitoring> </cs:context> </wsag:Context> <wsag:Terms> <wsag:All> <wsag:GuaranteeTerm wsag:Name="NinjaInvoiceHourlyRamLessEqual70PercentageConditionGuaranteeTerm"> <wsag:ServiceLevelObjective> <wsag:KPITarget> <wsag:KPIName>NinjaInvoiceHourlyRamLessEqual70PercentageConditionGuaranteeTerm_kpi</wsag:KPIName> <wsag:CustomServiceLevel> <cs:slo> <cs:constraint>NinjaInvoiceHourlyRamCompositeMetricContext LE 70.000000</cs:constraint> <cs:conditionConstraint>NinjaInvoiceHourlyRamLessEqual70PercentageCondition NOT_EXISTS</cs:conditionConstraint> <cs:description>the mean of ram of ninja invoice in 1 hours should be always less or equal than 70%</cs:description> </cs:slo> </wsag:CustomServiceLevel> </wsag:KPITarget> </wsag:ServiceLevelObjective> <wsag:BusinessValueList> <wsag:CustomBusinessValue count="1" duration="P1D"> <sla:Penalty type="discount" expression="10.00" unit="eur" validity="P1M"/> <sla:description>if 1 violation occurs in 1 day => apply a discount of 10€ for the current month</sla:description> </wsag:CustomBusinessValue> </wsag:BusinessValueList> </wsag:GuaranteeTerm> <wsag:GuaranteeTerm wsag:Name="NinjaInvoiceHourlyResponseTimeLessEqual20msConditionGuaranteeTerm"> <wsag:ServiceLevelObjective> <wsag:KPITarget> <wsag:KPIName>NinjaInvoiceHourlyResponseTimeLessEqual20msConditionGuaranteeTerm_kpi</wsag:KPIName> <wsag:CustomServiceLevel> <cs:slo> <cs:constraint>NinjaInvoiceResponseTimeCompositeMetricContext LE 20.000000</cs:constraint> <cs:conditionConstraint>NinjaInvoiceHourlyResponseTimeLessEqual20msCondition NOT_EXISTS</cs:conditionConstraint> <cs:description>the mean of response time in 1 hours should be always less or equal than 20ms</cs:description> </cs:slo> </wsag:CustomServiceLevel> </wsag:KPITarget> </wsag:ServiceLevelObjective> <wsag:BusinessValueList> <wsag:CustomBusinessValue count="1" duration="P1D"> <sla:Penalty type="discount" expression="10.00" unit="eur" validity="P1M"/> <sla:description>if 1 violation occurs in 1 day => apply a discount of 10€ for the current month</sla:description> </wsag:CustomBusinessValue> </wsag:BusinessValueList> </wsag:GuaranteeTerm> </wsag:All> </wsag:Terms> </wsag:Agreement>

Further analysis

This section elaborates about our research on some additions to the specification that were not finally implemented on the prototype. Special thanks to Kyriakos Kritikos (FORTH).

Cost model

The following information can be expressed for a cost model. The proposed structure is:

Pricing

minPrice: minimum price threshold. The price can never go below this threshold, irrespectively of the number of discounts that are legitimate to be claimed by the service customer.
maxPrice: maximum price threshold. Similarly, the service requester is guarded by this threshold in case of rewards such that he/she will never have to pay more than this threshold for a certain billing period.
basePrice: base price for the service. This is the normal price not affected by any discount or reward.
priceUnit: the currency for the service price (e.g., euros, dollars, etc.).

Charging

Charging frequency (duration - every 1 hour, 1 day, 1 month, etc)
Charging description (human-oriented description of what is charged - could be more elaborate on a technical level to explain how the price is calculated - could also provide links to web pages where this information is much detailed)
Reservation type: on-demand, advanced reservation, spot prices

Billing

Billing frequency (duration - every month, every 6 months, every year, etc.)
Billing means (credit card, bank transfer, paypal)
Billing process description (human-oriented description of how billing should be performed)

The excerpt below shows an example of a cost model expressed in WS-Agreement.

 <cs:priceModel name="Service Price Model"> <cs:pricing maxPrice="90" minPrice="40" basePrice="0.1" priceUnit="euros"/> <cs:charging frequency="1H" reservationType="on-demand"> <cs:description>service is charged every 1 hour of use</cs:description> </cs:charging> <cs:billing frequency="1M" means="credit-card,paypal"> <cs:description>the customer has to first provide the credit card details which will then be charged based on the charging frequency according to the actual cost incurred in the billing period</cs:description> </cs:billing> </cs:priceModel>

Smart Business-IT Alignment in the Cloud 1.2

2017-07-24T15:19:54Z

Business requirements

1.Smart and intuitive service discovery

2.Matching business process requirements with cloud service specifications

3.Ensuring consistency between human and machine interpretation of BPaaS models

4.Support business users to formulate decision logic for the Business-IT alignment

Solutions

1) BPaaS Design Environment Prototype

Extending BPMN 2.0x

-Business process requirements

-Workflow/cloud service specifications

Semantic Lifting

-Translating models into BPaaS Ontology

-Web service enabling semantic annotation

Matching

-Mapping business language to technical language

-Comparing business process requirements with cloud service specifications

2) Context-Adaptive Questionnaire Prototype

-Hierarchical service discovery: process – group – task

-Ask only relevant question about business process requirements

-Intelligent, dynamic prioritization of questions

3) DMN to SPARQL CONSTRUCT Prototype

-Decision tables for mapping of business process requirements to cloud service specifications

-Model decision rules with reference to BPaaS Ontology

-Translation of decision table into executable SPARQL Constructs

Materials for the solutions

Link to the video for installation and running of above prototypes

https://www.youtube.com/watch?v=f_9i6fSlc_A&t=199s

Design Environment Components 1.6

2017-07-17T10:08:59Z

1 Summary
2 Architecture design
3 Installation manual
- 3.1 Development
- 3.2 Production
4 Test Cases
5 User Manual
6 API specification
7 HandBook

Summary

The BPaaS Design Environment provides appropriate conceptual modelling tools for (a) designing domain specific business processes, (b) executable workflows, (c) additional description and rules for deployment as well as (d) Key Performance Indicators.

In order to provide those different modelling tools within one environment, a meta modelling platform is used that enables the plug-in of different modelling aspects. For BPaaS the aforementioned different modelling approaches correspond to the first two business process layers – the (I) domain specific business process as well as the (II) executable workflows – and the alignment from the business to the technical layer.

Hence the meta modelling platform enables to keep all models in one repository and the interaction between the different layers via so-called model weaving and semantic lifting.

The BPaaS Design Environment consists of:

A meta modelling platform that provides a BPaaS model repository for all its models, the corresponding management and security infrastructure and a development environment that enables the implementation of modelling components.
BPaaS modelling components are distinguished by their modelling languages – which are implementations of standards like BPMN, DMN, or RDF – as well as modelling features such as user interaction and model processing. Hence the domain specific business process modeller and the executable workflow modeller are both such a meta modelling component.
The corresponding Web-GUI realizes the user interaction features, to manipulate a model.
Interfaces enable the access to the BPaaS Design Environment in particular to the BPaaS model repository, which consists of domain specific business process models, executable workflow models, additional business requirement models and KPI models. This interaction can rely on standard features such as BPMN export / import or on implemented proprietary exchange formats.

Architecture design

The architectural design is available in D4.1 (available here). All UML diagrams and artifacts can be accessed online here.

Installation manual

Development

Production

Test Cases

User Manual

API specification

HandBook

Evaluation Environment Components 2.1

2017-05-25T13:52:08Z

Overview

The BPaaS Evaluation Environment is a companion to the CloudSocket broker which assists him/her towards optimising the BPaaS services offered. This assistance comes in various forms that map to different types of analysis. These types include: (a) KPI evaluation for assessing the performance of the BPaaS; (b) KPI drill-down to find the root causes of a particular KPI violation; (c) best BPaaS deployment discovery to infer the best deployments for a certain BPaaS; (d) process model recreation via process mining. On top of this functionality there exists a Hybrid Business Dashboard which enables the broker to appropriately invoke the analysis functionality as well as to visualise the analysis results by employing the best possible visualisation metaphors. This dashboard is shared between the BPaaS Desing and Evaluation Environments highlighting the close relationship between these environments where the results of one environment (i.e., evaluation) can enable to optimise the main artefacts of the other (i.e., design). This also enables the fast switch from one to the other environment when the need for optimisation is spotted and has to be applied at once. However, we should also highlight that the results of one analysis kind, i.e., the best BPaaS deployment discovery, can be exploited also by the BPaaS Allocation Environment towards the quest to find the most optimal deployment for a certain BPaaS.

Each analysis functionality is offered by a composite component in the form of an engine. In this respect, KPI evaluation and drill-down are offered by the Conceptual Analytics Engine, best BPaaS deployment discovery is offered by the Deployment Discovery Engine, while the capability to execute process mining algorithms is offered by the Process Mining Engine. To support the proper operation of all these engines with respect to the analysis tasks that have to be performed, another engine called Harvesting Engine was developed with the main goal to harvest information from different information sources (i.e., components) in the CloudSocket architecture, to semantically uplift it with the assistance of two ontology (Evaluation and KPI OWL-Q Extension) and to finally store it in the Semantic Knowledge Base. The latter has been realised in the form of an API which has been built on top of an existing triple store.

The BPaaS Evaluation Environment, similarly to the BPaaS Design one, supports multi-tenancy out of the box. This has been mainly realised thanks to the Harvesting Engine which is able to store the information harvested and semantically uplifted in broker-specific fragments (i.e., RDF graphs) inside the Semantic Knowledge Base. This enables the system to service a broker by considering only the fragment that belongs to him/her, thus also allowing all analysis functionalities to focus only on a specific portion of the whole Semantic Knowledge Base content.

As indicated in the analysis of each individual main component of the BPaaS Evaluation Environment, in many cases most of the components can be interchanged with external ones developed by organisations out of the CloudSocket consortium. This is due to the loose coupling between the components, the development of an abstraction API as well as the interchange of standard, as much as possible, formats. Two main exceptions apply to this rule: (a) the Hybrid Business Dashboard component is shared between the two environments which means that if it has to be interchanged, this should also be allowed by the BPaaS Design Environment; (b) the Harvesting Engine does not offer a certain REST API but is executed periodically in the form of a thread. This engine requires high expertise and knowledge of semantic technologies in order to be reproduced as well as a good knowledge of many APIs that are offered by the CloudSocket platform. It also needs to know what is the information offered by such APIs and how it can be correlated in such a way that appropriate semantic relationships between the different resources created can be expressed by considering the current schema of the Evaluation and KPI (OWL-Q Extension) ontologies. In this respect, we consider the development of such a component a difficult but not impossible job to be performed.

An overview of the architecture of the BPaaS Evaluation Environment is depicted in the figure below. As it can be easily seen, this architecture comprises three main levels: (a) UI; (b) business logic; (c) data. At the UI level resides the Hybrid Business Dashboard responsible for invoking the different analysis components offered. The business logic level comprises all the analysis engines (Conceptual Analytics Engine], [[Deployment Discovery Engine, and Process Mining Engine) as well as the Harvesting Engine. On top of the analysis engines resides the Hybrid Business Management Tool which takes care of the orchestration of the analysis functionality and is the bridge between the UI and the business logic level. Finally, at the data level, the Semantic Knowledge Base exists which enables the querying and the management of the semantic information stored in it.

Architecture of the BPaaS Evaluation Environment.

More information about this architecture, how the components interact with each other at the same or across different adjacent levels and what are the interactions scenarios covered can be found at the following URL: where corresponding UML diagrams can be viewed. This information was also covered in the D4.5 deliverable.

Below, we provide the links of all the components of the BPaaS Evaluation Environment which provide a more thorough analysis for them catering for different types of audience, including developers and operation managers. An overview of these components is also provided in the D4.6 deliverable.

Component Documentations

Semantic Knowledge Base

Harvesting Engine

Conceptual Analytics Engine

Process Mining Engine

Deployment Discovery Engine

Conceptual Analytics Engine 7.6

2017-05-25T13:13:59Z

1 Summary
2 Component responsible
3 Architecture Design
4 Installation Manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 API Specification
- 6.2 Handbook

Summary

The Conceptual Analytics Engine is a composite component in the BPaaS Evaluation Environment which is responsible for performing two types of KPI analysis over the Semantic Knowledge Base (Semantic KB) at the Data Layer of this environment's architecture. This component is invoked by the Hybrid Dashboard to execute the required type of analysis and visualise the respective analysis results according to the best possible visualisation metaphor. The two types of KPI analysis include:

KPI metric measurement: Derives the current and historical measurements for a KPI metric enabling their evaluation against the corresponding KPI threshold by the Hybrid Business Dashboard. This type of analysis is offered in two forms: (a) on-the-fly derivation of the KPI measurements based on measurement information of raw metrics derived from the BPaaS Execution Environment; (b) querying over a historical fragment of the Semantic KB to obtain the measurement history of the KPI. The first form is actually a facilitator of the second as it enables the broker to store the derived information in the historical fragment. The first form is also offered in terms of two variants: (a) KPI measurement: the measurement of the metric of an existing KPI is supported; (b) Metric measurement: measurement for metric specifications which do not necessarily map to KPIs. The second variant is ideal for exploring the possible metric space in order to enable the broker to come up with new KPIs for his/her BPaaSes. This has been deemed a necessity for KPI analysis environments as the determination of KPIs is not an easy task to perform which also requires some form of creativity that has to be supported by such an environment.
KPI drill-down: Drills-down the measurement of a metric of a KPI to the whole KPI hierarchy. Via the checking of the corresponding thresholds of the KPIs involved in the hierarchy, this enables to perform root-cause analysis (for the KPI violation that has occurred at the top-level of the hierarchy).

The Conceptual Analytics Engine does support multi-tenancy. This means that it can support multiple brokers instead of one in a concurrent manner. This feature is enabled via the Harvester component which partitions accordingly the information space in the Semantic KB in broker-specific fragments. Each broker-specific fragment is further separated into normal and historical sub-fragments. In this way, an analysis functionality for a certain broker can be operated over the suitable broker-specific fragment such that, in the end, multiple tenants/brokers can be supported.

Apart from API methods that directly map to the core KPI analysis functionality offered, also auxiliary/utility methods are also offered to better facilitate the analysis and visualisation tasks of the Hybrid Business Dashboard. In overall, the following functionality is offered by the Conceptual Analytics Engine component:

KPI Measurement: the capability to measure the performance of a BPaaS is offered either for the current moment or across a certain time period. The measurement can be across all customers of the broker that have purchased the corresponding BPaaS or for a certain customer. The measurement can be performed either via the KPI name, for fixed KPIs originally defined for the BPaaS at hand, or via the specification of a KPI metric (for a more creative exploration of the possible metric space).
KPI Drill-Down: this offered capability enables to perform root-cause analysis for the violation of a top KPI. As soon as a top KPI is violated, the Hybrid Business Dashboard can call this functionality in order to check the status of the KPIs within the hierarchy of the top-level KPI violated. What happens in the background is that both the top-level KPI and its descendants are evaluated via the first type of functionality analysed in the previous bullet. The measurement results are then compared with the KPI thresholds to derive the chain of KPI violations that led to the top-level KPI violation.
Retrieval of Raw Metric Information: this capability enables to obtain the information of which raw metrics can be exploited by the broker in its effort to produce a new KPI metric. Such raw metrics map to measurements that have been already collected from the BPaaS Execution Environment by the Harvester. These metrics are actually exploited in the production of the KPI metric hierarchy mapping to a new KPI metric that has to be evaluated by the broker via the invocation of the functionality in the 1st bullet. Thus, they participate in the formulas of the metrics involved in this KPI metric hierarchy.
Tenant Information Retrieval: this capability enables the broker to obtain the list of tenants for which KPI measurements have been produced for a certain BPaaS. Such information can then be exploited, e.g., in order to perform KPI measurement and drill-down in a customer-specific manner.
KPI Retrieval: could be argued that this capability is not needed to be offered as the Hybrid Business Dashboard already has the knowledge about which KPIs have been defined for a BPaaS inside the BPaaS Design Package. However, by considering the fact that some KPIs are defined later on by the BPaaS Evaluation Environment, this utility capability, in the end, enables to present to the broker all KPIs that have been defined for a BPaaS and have been incorporated in the corresponding BPaaS bundle definition.

Only the CloudSocket broker has been enabled to exploit such the Conceptual Analytics Engine with the rationale that the BPaaS Evaluation Environment is a facility offered only for this entity that enables it to assist it in the proper management of its BPaaSes. Of course, any CloudSocket enhanced implementation might also take the view of providing an analysis functionality also for the customer. In any case, we consider that only one role from the CloudSocket broker can be involved in the exploitation of the BPaaS Evaluation Environment which is the evaluation expert. This is due to the fact that both the Hybrid Business Dashboard as well as the requirements imposed by its underlying analysis functionality do not force the use of a more technical role. In the context of KPI analysis, this is in contrast to other frameworks which tightly couple the definition of KPIs with the underlying measurement/storage technology (e.g., database queries used for KPI evaluation). On the contrary, the Conceptual Analytics Engine does not impose such a requirement as it demands the use of the OWL-Q language, which is a high-level, implementation-independent language for the definition of KPIs. In this way, the evaluation expert needs just to learn this language in order to define new KPIs which comprise metrics that are specified in a quite user-intuitive and familiar way for the expert. In case that new KPIs are not needed to be specified, then the evaluation expert does not need to possess any special knowledge in order to exploit the analysis functionality offered which greatly eases his/her job.

This component could be easily replaced by another component developed by an organisation outside the CloudSocket consortium. Its integration will be easy as it is loosely coupled with just one from the rest of the BPaaS Evaluation Environment components, the Semantic KB and especially its Semantic KB Service which hides and abstracts away from any implementation details involved in the Semantic KB.

The following table indicates the details of the component.

Type of ownership	Creation
Original tool	New component developed in the context of this project
Planned OS license	Mozilla Public Licence 2.0.
Reference community	ADOxx Community.

Consist of

Conceptual Analytics Service (REST API exposing all core functionalities)

Depends on

Semantic KB

Component responsible

Developer	Email	Company
Kyriakos Kritikos	kritikos@ics.forth.gr	FORTH

Architecture Design

The Conceptual Analytics Engine is a composite component offering KPI analysis capabilities, which is coloured in red in the overall BPaaS Evaluation Environment architecture depicted in the figure below. As it can be seen, it is invoked by the Business Management Tool in order to execute the respective BPaaS analysis functionality. This invocation, in turn, leads to exploiting the content of the Semantic KB at the Data Layer in order to support the analysis functionality needed.

The final, internal architecture of the Conceptual Analytics Engine, which is depicted below, comprises the following 6 main components:

Conceptual Analytics Service: A REST service which encapsulates the whole functionality of the Conceptual Analytics Engine in the form of REST API methods. Depending on the respective API method invoked, different underlying components are executed by this service.
Query Creator: this component is responsible for creating SPARQL queries which are then evaluated by the Conceptual Analytics Service over the Semantic KB. Such queries map to the core functionality of the Conceptual Analytics Engine, thus enabling to obtain measurements related to KPI assessment and drill-down. They also map to auxiliary/utility functionality which is also offered in the form of specific API methods of the Conceptual Analytics Service.
Drill-Down Handler: this component is responsible for performing the drill-down of KPIs. As a KPI forms a hierarchy with its underlying KPI descendants, the facilities of the Query Creator are exploited in order to construct the KPI measurement queries for all the KPIs involved in the KPI hierarchy.
KPI Handler: this component is responsible for obtaining the definition of KPIs and their corresponding metrics from the Semantic KB and offers the functionality of retrieving such information both to the Conceptual Analytics Service and the two main components underlying it. This means that there exist API methods which can be exploited in order to obtain the KPI information involved in the definition of a certain BPaaS.
SPARQL Transformer: it is responsible for transforming the definition of a KPI metric into an SPARQL query which can be exploited in order to derive the measurements of that metric in a certain time period. It is solely exploited by the Query Creator component.
Resource Accessor: it has the duty to retrieve information from external information sources, i.e., REST APIs or databases. It is called by the Query Creator in order to produce the appropriate input information (mapping to such external information source accessing) in metric formulas which is to be implanted in the query that has to be generated for a specific metric. This is due to the fact that metric formulas comprise input parameters, which can be metrics, constants or information generated from calling such external information sources.

Architecture of the Conceptual Analytics Engine.

Installation Manual

The component has been implemented purely in java. It depends on other external components / frameworks like Spring, swagger, jersey and sesame/rdf4j. It also depends on an internal component developed in the context of this project, the owl-q one which enables the parsing of OWL-Q models into domain-specific objects as well as their writing. The following figure explicates the sole artefact generated mapping to this component as well as its dependencies on other artefacts (at a coarse-grained level to simplify the figure).

Artifact dependencies for the Conceptual Analytics Engine.

Maven has been exploited as the main build automation tool, prescribing how the component/artefact can be built and its main dependencies. Such a tool facilitates the building and automatic generation of this artefact for the different CloudSocket environments (development and production) and their configuration. A complete description of how such artefact building and generation can be performed for the different environments is given in the installation manual below.

Development

Currently, the following requirements hold for this component:

Oracle's 1.7.x JDK or higher
Apache tomcat 1.7 or higher
Maven tool for code compilation and packaging

The installation procedure to be followed is the one given below:

Download source code from https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/evaluation_skb/repository/archive.tar?ref=master
Unzip code with tar (or any other tool)
Go to the root directory of the installed code
Change the configuration file to provide the right access information for the Virtuoso Triple Store (https://virtuoso.openlinksw.com/)
Run:
```
 mvn clean install 
```
and then:
```
 mvn war:war 
```
Move the war file to the webapps directory of tomcat and start tomcat, if not yet started
Test installation by entering in your browser the following URL: http://localhost:8080/evaluation/

Production

The same instructions as above hold for this component

Test Cases

You can run test cases directly from the URL (http://localhost:8080/evaluation in your development environment) of the REST service due to the use of swagger (http://swagger.io) which enables the execution of the API method exposed by exploiting user input provided in a form-based manner. The following test cases are envisioned:

Run a KPI evaluation query

Suppose that the user desires to obtain the most recent value for a KPI named as "MeanAvailabilityKPI". Then, he/she can browse the methods of the API and click on the one named as "evaluateKPI". The following screenshot indicates the upper part of the description of the method when selected: Next, he/she can fill in the kpiName field with the value of "MeanAvailabilityKPI", the field of "brokerId" with the value of "bwcon" and the field of "bundleId" with the value of "SendInvoiceSaaSEurope". Then he/she can press the "Try it out!" button.

Finally, he/she will be able to see the respective curl command issued, the request URL, the corresponding results, as well as the response status and headers as indicated in the following figure:

User Manual

API Specification

The reader should refer to the swagger-based API web page (http://localhost:8080/evaluation for a local installation or http://134.60.64.222:8080/evaluation for a remote existing installation) for browsing an on-line documentation with the capability to execute the API methods. In the following, the API methods of the Conceptual Analytics Service are analysed below:

evaluateKPI

This method enables to produce current and historical measurements for a certain KPI in the context of a specific BPaaS. As multi-tenancy is supported, the id of the broker needs to be specified in order to set the appropriate context for the method execution.

 POST ca/evaluateKPI/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the evaluation needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the evaluation needs to be performed (optional)
kpiName: denotes the name of the KPI to be evaluated (obligatory)
kpiPeriod: denotes the (history) period of the evaluation. If this parameter is specified, then a set of KPI evaluation values and not just one can be returned matching the respective period provided. The value should conform to String-based Java periods (e.g., "P1M" which declares a period of 1 month until now) (optional)
tenant: denotes the id of the tenant for which the KPI can be evaluated. In this way, the KPI is assessed only for this tenant and not across all tenants that have purchased the respective BPaaS (optional)
maxRows: denotes the maximum number of KPI values to return. Default value is 0 indicating that all possible values can be returned (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in both json and xml formats. A sample for a json output is given below:

 { "results": [ { "bundleId": "SendInvoiceSaaSEurope", "value": "0.8010364041400798e0", "instant": "2016-01-27T11:43:38.588Z" } ] }

evaluateMetric

This method enables the production of current and historic measurements for a KPI metric. It is similar to the "evaluateKPI" method but requires from the broker to appropriately specify the KPI metric in two different ways: (a) the metric and its context are passed as input parameters; (b) the name of an existing metric is provided:

 POST ca/evaluateMetric/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the evaluation needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
metricName: denotes the name of the existing metric for which the evaluation needs to be performed. This parameter is optional as there is an alternative way of determining the KPI metric to be evaluated (optional)
metric: provides a CompositeMetric specification in OWL-Q for the metric to be evaluated. As a composite metric is associated to a specific metric context, the "metricContext" parameter does not need to be provided in this case (optional)
metricContext: provides the MetricContext specification in OWL-Q for the metric to be evaluated. This metric context includes scheduling and window information for the metric at hand. This parameter is optional and might be provided in case that we need to modify the context of an existing metric (whose name is specified in the "metricName" parameter) temporarily for the context of this evaluation (optional)
bundleId: denotes the id of the BPaaS bundle for which the evaluation needs to be performed (optional)
kpiPeriod: denotes the (history) period of the evaluation. If this parameter is specified, then a set of metric measurement values and not just one can be returned matching the respective period provided. The value should conform to String-based Java periods (e.g., "P1M" which declares a period of 1 month until now) (optional)
tenant: denotes the id of the tenant for which the identified metric can be evaluated. In this way, the metric measurements are only produced for this tenant and not across all tenants that have purchased the respective BPaaS (optional)
maxRows: denotes the maximum number of metric measurements values to return. Default value is 0 indicating that all possible values can be returned (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in both json and xml formats. A sample for a json output is given below:

 { "results": [ { "bundleId": "SendInvoiceSaaSEurope", "value": "0.8010364041400798e0", "instant": "2016-01-27T11:43:38.588Z" } ] }

kpiDrillDown

This method enables to perform a drill-down over a certain KPI for a specific BPaaS (again under the context of a specific broker). The drill-down results can be provided in a tree-based form in both JSON & XML formats, where each result contains information which is equivalent to that of the aforementioned methods:

 POST ca/kpiDrillDown/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the KPI drill-down needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the KPI drill-down needs to be performed (optional)
kpiName: denotes the name of the KPI for which the KPI drill-down needs to be performed (obligatory)
kpiPeriod: denotes the (history) period of the drill-down. If this parameter is specified, then a set of drill-down values and not just one can be returned matching the respective period provided. The value should conform to String-based Java periods (e.g., "P1M" which declares a period of 1 month until now) (optional)
tenant: denotes the id of the tenant for which the identified KPI needs to be drilled down. In this way, the respective KPI measurements and their drill-down are only produced for this tenant and not across all tenants that have purchased the respective BPaaS (optional)
maxRows: denotes the maximum number of drill-down values to return. Default value is 0 indicating that all possible values can be returned (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in both json and xml formats. A sample for a json output is given below:

 { "results": [ { "bundleId": "SendInvoiceSaaSEurope", "kpi": "MeanKPIAvailability", "value": "0.8010364041400798e0", "instant": "2016-01-27T11:43:38.588Z", "subRows": [ { "bundleId": "SendInvoiceSaaSEurope", "kpi": "MeanIaaSKPIAvailability", "value": "0.8526374638629035e0", "instant": "2016-01-27T11:43:37.466Z", "subRows": [] } ] } ] }

kpiHistory

This method enables to query the historical measurements for a certain KPI in the context of a specific BPaaS. As multi-tenancy is supported, the id of the broker needs to be specified in order to set the appropriate context for the method execution. The results are equivalent to those returned by the "evaluateKPI" method. Please note that the content of the history fragment of the broker concerned needs to be populated via the "evaluateKPI" method in order to be able to obtain any result.

 POST ca/kpiHistory/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the historical query over the broker-specific historical fragment needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the historical query needs to be performed (optional)
kpiName: denotes the name of the KPI for which historical measurements need to be obtained (obligatory)
kpiPeriod: denotes the (history) period. If this parameter is specified, then a set of KPI historical measurement values and not just one can be returned matching the respective period provided. The value should conform to String-based Java periods (e.g., "P1M" which declares a period of 1 month until now) (optional)
tenant: denotes the id of the tenant for which the KPI history query can be evaluated. In this way, the required historical KPI values to be retrieved map only to this tenant and do not cross all tenants that have purchased the respective BPaaS (optional)
maxRows: denotes the maximum number of KPI historical measurement values to return. Default value is 0 indicating that all possible values can be returned (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in both json and xml formats. A sample for a json output is given below:

 { "results": [ { "bundleId": "SendInvoiceSaaSEurope", "value": "0.8010364041400798e0", "instant": "2016-01-27T11:43:38.588Z" } ] }

kpiQuery

This method enables to retrieve the list of KPIs that have been defined in the context of a specific broker. When the id of the BPaaS is also provided, then the KPI list is filtered to include only those KPIs that have been specified for the BPaaS.

 POST ca/kpiQuery/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the KPI query needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the defined KPIs need to be retrieved - as stated, if not given, then this maps to obtaining all the KPIs that have been defined for all BPaaSes of a certain broker (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in both json and xml formats. A sample for a json output is given below:

 [ { "name": "MeanIaaSAvailabilityKPI", "description": null, "abbreviation": null, "synonym": null, "constraintContext": null, "obliged": null, "firstArgument": { "name": "MeanIaaSAvailability", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#IaaS1", "attribute": null, "level": 2, "metricContext": { "name": "MC1", "description": null, "abbreviation": null, "synonym": null, "window": null, "schedule": { "name": "OneDaySchedule", "description": null, "abbreviation": null, "synonym": null, "interval": 1, "repetition": 0, "scheduleType": "FIXED_RATE", "timeUnit": { "name": "Day", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": null, "quantity": null, "quantityKind": null, "unitType": "SINGLE", "unitMappings": null, "uri": "http://www.cloudsocket.eu/evaluation#Day" }, "uri": "http://www.cloudsocket.eu/evaluation#OneDaySchedule" }, "uri": "http://www.cloudsocket.eu/evaluation#MC1" }, "formula": { "name": "mean_iaas_availability", "description": null, "abbreviation": null, "synonym": null, "function": "MEAN", "functionURI": null, "argumentList": [ { "RawMetric": { "name": "IaaSAvailability", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#IaaS1", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/IaaSAvailability" } } ], "uri": "http://www.cloudsocket.eu/formulas/mean_iaas_availability" }, "metricList": [ { "RawMetric": { "name": "IaaSAvailability", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#IaaS1", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/IaaSAvailability" } } ], "uri": "http://www.cloudsocket.eu/metrics/MeanIaaSAvailability" }, "comparisonOperator": "GREATER_EQUAL_THAN", "optimisationOperator": null, "secondArgument": 0.9, "warningThreshold": 0.8, "validity": null, "childKPIs": null, "bundleURI": "http://www.cloudsocket.eu/evaluation/bwcon#SendInvoice", "qualifyingCondition": false, "uri": "http://www.cloudsocket.eu/kpis/MeanIaaSAvailabilityKPI" } ]

kpiTenantQuery

This method enables to retrieve the list of tenants/customers for which there exist measurements for a certain KPI for the BPaaSes that they have purchased in the context of a specific broker. When the id of the BPaaS is also provided, then the tenant list is filtered to include only those tenants which have purchased the respective BPaaS.

 POST ca/kpiTenantQuery/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the KPI tenant query needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle that must have been purchased by the tenants to be returned (optional)
kpiName: denotes the name of the KPI for which measurements should have been produced for the tenants to be returned (obligatory)
kpiPeriod: denotes the historical period for which KPI measurements should have been produced. The value should conform to String-based Java periods (e.g., "P1M" which declares a period of 1 month until now) (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported
500 -- Internal server error exception

The output can be in both json and xml formats. A sample for a json output is given below:

 [ { 3c1ab6ac-6a7e-4791-8c16-f8f6d400231c } ]

ldQuery

This method enables to pose an arbitrary query over the broker-specific fragment of the Semantic KB. Such a query should only be posed by an expert on semantic technologies which should have an appropriate knowledge of SPARQL as well as of the Evaluation and OWL-Q (KPI Extension) ontologies that have been developed in the context of WP3 in CloudSocket.

 POST ca/ldQuery/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the KPI tenant query needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
query: signifies the SPARQL query to be posed over the Semantic KB (obligatory)
timeout: specifies the maximum amount time for waiting (in milliseconds) until a response can be obtained from the Semantic KB. A value of 0, which is the default, denotes that the broker can wait indefinitely (optional)
maxRows: denotes the maximum number of rows to be produced by the query. A value of 0, which is the default, denotes that all rows should be returned. As SPARQL has the LIMIT construct, the broker should be careful of not specifying SPARQL queries including this construct that are evaluated via this method for which has a positive value for this parameter has been provided. In this latter case, the outcome would depend on the correlation between the values for the LIMIT construct and this parameter, i.e., if LIMIT is X and parameter value is Y, >= X, then X values will be returned. If Y < X, then Y values will be returned. In other words, the lowest from the values provided will prevail (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output can be in "application/sparql-results+xml", "application/sparql-results+json", "text/csv" and "text/tab-separated-values". A sample for the "application/sparql-results+json" output is given below:

 { "head": { "link": [], "vars": ["bpaas", "id"] }, "results": { "distinct": false, "ordered": true, "bindings": [ { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceSaaSEurope" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceSaaSEurope" }}, { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceSaaSWorldwide" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceSaaSWorldwide" }}, { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceIaaSEurope" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceIaaSEurope" }} ] } }

rawMetricQuery

This method enables to obtain all the raw metrics that have been defined in OWL-Q for a certain broker. In case that a respective BPaaS is also determined, then the raw metric list is filtered to contain only those raw metrics that have been defined for this BPaaS. As has been already stated, this method is a utility one which enables the broker to construct KPI metric specifications out of those returned by this method in order to subsequently make evaluation queries for those KPI metrics via the "evaluateMetric" method and thus better explore the possible KPI metric space.

 POST ca/rawMetricQuery/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the raw metric query needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle in order to restrain the raw metric list to be obtained to contain only those metrics that have been defined for this bundle (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found

The output can be in XML or JSON form. A sample of an output in JSON form is given below:

 [ { "name": "CycleTime", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "NEGATIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#Workflow_SendInvoice", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/CycleTime" }, { "name": "EmailNotSent", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "NEGATIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#Workflow_SendInvoice", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/EmailNotSent" }, { "name": "EmailSent", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#Workflow_SendInvoice", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/EmailSent" }, { "name": "SaaSAvailability", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#ExternalSaaS1", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/SaaSAvailability" }, { "name": "CustomerSatisfaction", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#Workflow_SendInvoice", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/CustomerSatisfaction" }, { "name": "IaaSAvailability", "description": null, "abbreviation": null, "synonym": null, "scale": null, "monotonicity": "POSITIVE", "unit": null, "valueType": null, "aggregationType": null, "measuredObject": "http://www.cloudsocket.eu/evaluation#IaaS1", "attribute": null, "level": 1, "metricContext": null, "sensor": null, "measurementDirective": null, "manual": true, "uri": "http://www.cloudsocket.eu/metrics/IaaSAvailability" } ]

Handbook

Harvesting Engine 5.5

2017-05-25T13:13:27Z

1 Summary
2 Component responsible
3 Architecture Design
4 Installation Manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 Handbook

Summary

The Harvesting Engine is a composite component in the BPaaS Evaluation Environment which is dedicated to:(a) the collection of deployment, monitoring and registry information from the BPaaS Execution Environment and the Repository Manager; (b) the semantic transformation and linking of that information; (c) the storage of that information in the Semantic KB.

While this component is independent of the analysis components in the BPaaS Evaluation Environment, it actually enables them as it provides the proper information via which the corresponding analysis can be performed. To this end, by also considering that the information to be collected does not need to be harvested very frequently in order not to overwhelm the environment, this component is run periodically in order to perform its harvesting, semantic linking and storage job in the form of a Java thread.

The Harvesting Engine is the component which also enables multi-tenancy in the BPaaS Evaluation Environment. As the sole user of that environment is the CloudSocket broker, multi-tenancy is offered in the form of the capability to simultaneously handle multiple brokers. This is achieved by the careful storage of the harvested information into disjoint, broker-specific fragments in the Semantic KB that only the corresponding broker can see and reason about. To this end, by knowing exactly which broker makes the respective analysis request, the corresponding analysis components in the BPaaS Evaluation Environment can focus their analysis only on the fragment attributed to the broker concerned.

The information harvesting is performed by invoking the REST APIs offered by corresponding components in the CloudSocket platform. As such, the information sources currently exploited span: (a) the Workflow Engine (for collecting workflow modelling, task-to-service deployment & instance-based information); (b) the Cloud Provider Engine (for collecting service-to-IaaS deployment & monitoring definition information); (c) the Monitoring Engine (for collecting produced monitoring information); (d) the Repository Manager (for collecting information from the Abstract Service, Concrete Service and Software Component Registries).

The collected information is semantically transformed and linked by conforming to the Evaluation and OWL-Q (KPI Extension) ontologies. This means that the information is triplified according to the schema of these ontologies by also handling the corresponding links between the rdf resources generated for cross-points between the two ontologies. Such linking enables to connect deployment and monitoring information in order to allow the posing of suitable KPI metric evaluation queries which aggregates measurements only for those BPaaS elements that are correlated with a certain deployment hierarchy.

This component is not easily replaceable as it is the glue point between BPaaS evaluation-related and many other CloudSocket components in other environments. As such, the internal logic of that component could be quite hard to reproduce unless there is a deep knowledge of all the components from which this component has to extract information. In addition, there is the need to possess a good level of expertise in semantic technologies to realise this component due to the requirement to structure and link the harvested information based on the two ontologies considered.

The following table indicates the details of the component.

Type of ownership	Creation
Original tool	New component developed in the context of this project
Planned OS license	Mozilla Public Licence 2.0.
Reference community	ADOxx Community.

Consist of

Harvester (main orchestration component run in the form of a Java thread)

Depends on

Semantic KB
Workflow Engine
Cloud Provider Engine
Monitoring Engine
Repository Manager (only the aforementioned registries which are directly accessed via their REST API)

Component responsible

Developer	Email	Company
Kyriakos Kritikos	kritikos@ics.forth.gr	FORTH

Architecture Design

The Harvesting Engine is coloured in purple in the overall BPaaS Evaluation Environment architecture depicted in the figure below. As it has been already stated, this is an independent component (inside the BPaaS Evaluation Environment) which only communicates with the Semantic KB in order to store the information collected and semantically lifted.

The final, internal architecture of the Harvesting Engine, which is depicted below, comprises 9 main components:

Harvester: This is the main component that coordinates the harvesting of the information to be collected from the BPaaS Execution Environment and the Repository Manager. Such a harvesting follows a 3 step process: (a) first IaaS service & cloud information is collected from the Cloud Provider Engine; (b) then workflow, deployment and monitoring definition information is collected from the Workflow Engine and the Cloud Provider Engine; (c) finally, monitoring information is collected from the Monitoring Engine. The component also takes care of how to enforce multitenancy by appropriately managing the broker-specific fragments and populating them with either general, cross-broker information or information that pertains only to the broker concerned.
5 components are extraction components which collect information from different CloudSocket components by calling their REST APIs. These components include: (a) the Deployment Extractor (deployment information collection); (b) the WF Engine Extractor (workflow-related information extraction); (c) Registry Extractor (registry information collection); (d) Monitoring Extractor (monitoring information extraction); (e) Marketplace Extractor (marketplace information collection -- not yet implemented).
BPMN Parser: takes care of parsing the BPMN model of the BPaaS workflow that has been collected from the WF Engine Extractor and of enforcing the triplification of the workflow-related information
CAMEL Parser: this component extracts the CAMEL model of the BPaaS workflow in order to obtain component and component-to-VM/IaaS deployment information as well as is capable of transforming the monitoring and KPI definition part of CAMEL into an OWL-Q model which is stored in the Semantic KB.
Triple Importer: this is the component which takes care of the triplification of the collected information, its semantic linking based on the schema of the two ontologies and the storage of this information in the Semantic KB

Architecture of the Harvesting Engine.

Installation Manual

The component has been implemented purely in java. It depends on other external components / frameworks like jersey (for making REST requests to the CloudSocket APIs) and sesame/rdf4j (for the semantic storage of the triplified information in the Semantic KB. It also depends on an internal component developed in the context of this project, the owl-q one which enables the parsing of OWL-Q models into domain-specific objects as well as their writing. The following figure explicates the sole artefact generated mapping to this component as well as its dependencies on other artefacts (at a coarse-grained level to simplify the figure).

Artifact dependencies for the Harvesting Engine.

Development

Currently, the following requirements hold for this component:

Oracle's 1.7.x JDK or higher
Maven tool for code compilation and packaging

The installation procedure to be followed is the one given below:

Download source code from https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/harvester/repository/archive.tar?ref=master
Unzip code with tar (or any other tool)
Go to the root directory of the installed code
Change the configuration file to provide the right connection information to all the information sources exploited
Run:
```
 mvn clean install 
```
to compile the source code
Run:
```
 mvn exec:java -Deu.cloudsocket.configdir=. 
```
to run the component as a Java thread which is configured by reading the "eu.cloudsocket.evaluation.dd.properties" configuration file from the current directory
As the component runs in command line mode, it is easy to check whether the component is running normally. In case that you need to examine the content that it generates, you should connect to the Triple Store of the Semantic KB and pose some SPARQL queries to draw and inspect it. This can be more easily done when the Triple Store is immediately empty.

Production

The same instructions as above hold for this component

Test Cases

As this component runs as a Java thread, it can be immediately inspected if it exhibits any bug (by, e.g., taking care of creating a log out of the printed output). In order to check also the content that it generates, a set of SPARQL queries can be posed over the Triple Store of the Semantic KB. If this triple store is developed locally, then you can connect to the local Virtuoso (Triple) Server endpoint (at http://localhost:8890/sparql) and then pose the queries from there, especially as it exhibits a nice interface to run SPARQL queries and visualise their results. If you desire to exhibit an existing endpoint, you can direct your queries at: http://134.60.64.222:8890/sparql.

The SPARQL queries, which act as test cases that can be posed and inspected if they produce a respective result, involve the following:

Get all BPaaS workflow tasks and their input and output parameters -- this query tests whether we can obtain correctly workflow information from the Workflow Engine and to correctly correlated it

 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?name ?id1 ?id2 from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?t eval:name ?name. { ?t a eval:ServiceTask } UNION { ?t a eval:ScriptTask } UNION {?t a eval:UserTask} optional{ ?t eval:inputVariable ?v1. ?v1 eval:id ?id1. } optional{ ?t eval:outputVariable ?v2. ?v2 eval:id ?id2. } }

Get user id as well as information about the roles assigned to the user (id, name and type) for the customers over all BPaaSes -- this query tests the ability of the Harvesting Engine to collect customer-specific information in the context of a BPaaS (workflow). The users collected map to the customer organisation but the roles are defined in the context of BPaaS workflow execution and not in general for the organisation. This means that Knowledge Worker can be a generic organisation role which would characterise a user responsible for the execution of user tasks in corresponding BPaaS workflow instances that is not currently captured but "accounting" can be a captured role which maps to a specific fragment of a BPaaS workflow denoting a person responsible for accounting-related tasks.
```
 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?name ?rid ?rname ?rtype from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?u a eval:User; eval:name ?name; eval:role ?r1. ?r1 eval:id ?rid; eval:name ?rname; eval:type ?rtype. } 
```
Get allocation information for each BPaaS which includes the workflow allocated, the allocation itself, the allocated task, the name of the (external or internal) SaaS service realising the functionality of the task, and the name of the IaaS service on which an internal SaaS service is deployed -- this information enables to check whether information from the Workflow Engine and the registries of the Repository Manager can be appropriately harvested and correctly correlated.
```
 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?bpaasId ?workflow ?alloc ?taskName ?serviceName ?iaasName from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?bpaas eval:allocation ?alloc; eval:id ?bpaasId; eval:workflow ?workflow. ?alloc eval:allocatedTask ?task; eval:selectedService ?service. ?task eval:name ?taskName. ?service eval:name ?serviceName. optional{ ?alloc eval:selectedIaaS ?iaas. ?iaas eval:name ?iaasName. } } 
```

Get information about all IaaS services -- this query tests whether we can obtain IaaS-related information from the Cloud Provider Engine

 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?iaas ?name ?coreNumber ?memorySize ?storageSize from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?iaas a eval:IaaS; eval:name ?name; eval:coreNumber ?coreNumber; eval:memorySize ?memorySize; eval:storageSize ?storageSize. }

Get information about all SaaS services -- This query tests whether we can successfully obtain and correlate SaaS-related information from the (Abstract/Concrete) Service and Software Component Registries of the Repository Manager

 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?service ?serviceName ?serviceOid ?serviceEndpoint from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?service rdf:type ?type; eval:name ?serviceName; eval:registryId ?serviceOid; eval:endpoint ?serviceEndpoint. FILTER(?type IN (eval:ExternalSaaS, eval:ServiceComponent)) }

Get information about the instances of a deployed workflow and its tasks -- This query tests whether we can retrieve instance-based information from the Workflow Engine and correctly link it to the deployed workflows and tasks already modelled.

 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?depWF ?depTask ?wfInstance ?taskInstance from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?depWF eval:workflowInstance ?wfInstance; eval:deployedTask ?depTask. ?depTask eval:taskInstance ?taskInstance. }

Get (not task-related) information about a workflow instance -- This query tests whether we can obtain workflow instance information from the Workflow Engine

 PREFIX eval: <http://www.cloudsocket.eu/evaluation#> SELECT ?wfInstance ?startTime ?endTime ?endState ?initiator from <http://www.cloudsocket.eu/evaluation/bwcon> WHERE { ?wfInstance rdf:type eval:WorkflowInstance; eval:startTime ?startTime; eval:endTime ?endTime; eval:endState ?endState; eval:initiator ?initiator. }

User Manual

Handbook

Deployment Discovery Engine 1.7

2017-05-25T13:12:38Z

1 Summary
2 Component responsible
3 Architecture Design
4 Installation Manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 API Specification
  - 6.1.1 deploymentDiscovery
- 6.2 Handbook

Summary

The Deployment Discovery Engine is a composite component in the BPaaS Evaluation Environment which is responsible for discovering best deployments for a BPaaS according to the deployment and measurement history of this and other similar or equivalent BPaaS to this BPaaS, as being stored in the Semantic KB. This component is invoked by the Hybrid Dashboard to execute this required type of BPaaS analysis and visualise the respective analysis results according to the best possible visualisation metaphor.

The discovery of best deployments currently relies on three main factors:

workflow equivalence/similarity: as an execution history of a BPaaS can be limited or non-existing as in the case of a new BPaaS, we need to find a way to explore the measurement and deployment history of other BPaaSes that are similar or equivalent to the current one. To this end, BPaaS similarity or equivalence needs to be formally defined and the means to achieve it realised. Our main way of addressing this was to rely on workflow similarity, i.e., compare the workflows of the BPaaSes concerned in order to infer their similarity. Workflow similarity, in turn, is mapped to task equivalence which is enforced by considering the task name. In other words, two workflows are equivalent if they include the same tasks -- we assume here that probably the structure of the workflow is similar although this might not be true in all possible cases. On the other hand, two workflows are similar if they share a certain percentage (75% currently but this can be configured) of equivalent tasks.
evaluation of "best" deployment: by considering that each BPaaS might have its own KPIs that need to be respected, it can be impossible to use a certain set of KPIs that can be common among many BPaaSes to evaluate whether a certain deployment can be considered as "best" or not. In order to remedy for this, we attempt to abstract away from KPIs by considering the overall level of QoS which can be considered to be derived via a weighted sum approach over utility functions constructed for each specific KPI metric exploited. In this sense, via such a normalisation and aggregation, each deployment then maps to just one overall QoS value and then ranking can be performed to infer which deployment is the best.
deployment selection: while it could be argued that each deployment of a BPaaS should be considered in the analysis, this can be quite risky as this might also map to taking into account deployments for which the KPIs have been violated. While this might not occur so frequently, it can still lead to economic loss for the broker and is thus subject for performing deployment filtering. This means that only deployments which have been successful in the past by not violating any KPI should be considered in the best deployment analysis. As such, the risk of violating a KPI would be the minimum possible while the probability of achieving a better or higher service level will be increased.

Based on the above analysis, there is no need for the user to provide a special input to this engine apart from providing the id of the BPaaS for which the analysis needs to be performed. In addition, the user can restrain the execution (measurement & deployment) history within a certain period with the rationale that possibly more recent periods might involve the execution of more optimised BPaaSes. The result of the best BPaaS deployment analysis takes the form of two tables which include: (a) the mapping of each service task to a certain SaaS (internal or external) and (b) the mapping of each internal SaaS to a certain IaaS. Currently, JSON is supported but in the future possibly DMN and/or CAMEL could also be included in the support list by also giving the user the opportunity to select the most appropriate form of the output (to check).

As enabled as a feature by the Harvesting Engine, the Deployment Discovery Engine does support multi-tenancy as it allows each broker to focus the analysis only on its own fragment in the Semantic KB. For this reason, the best deployment functionality is encapsulated in the form of a REST API with a single method which does take as input the id of the broker for which the analysis has to be performed plus optionally the history period for this analysis.

The Deployment Discovery Engine can be easily replaced by another component developed externally by an organisation outside the CloudSocket consortium. This is due to the fact that this component is not tightly coupled with any other internal component in the BPaaS Evaluation Environment. It just communicates with the Semantic KB in order to retrieve all the necessary information for inferring workflow equivalence and producing the rank of successful deployments, while it exposes an API to the Hybrid Business Dashboard in order to enable its functionality to be invoked. In this sense, any replacement component would just have to conform to that API as well as be able to draw the correct information from the content of the Semantic KB.

The following table indicates the details of the component.

Type of ownership	Creation
Original tool	Drools Knowledge Base Framework
Planned OS license	Mozilla Public Licence 2.0.
Reference community	ADOxx Community.

Consist of

Deployment Discovery Service (REST API exposing the core functionality of this engine)

Depends on

Semantic KB

Component responsible

Developer	Email	Company
Kyriakos Kritikos	kritikos@ics.forth.gr	FORTH

Architecture Design

The Process Mining Engine is is coloured in deep blue in the overall BPaaS Evaluation Environment architecture depicted in the figure below. As it can be seen, it is invoked by the Business Management Tool in order to start the best deployment analysis. This invocation, in turn, leads to exploiting the content of the Semantic KB at the Data Layer in order to retrieve the most suitable information needed to support this analysis.

The final, internal architecture of the Deployment Discovery Engine, which is depicted below, comprises the following 5 main components:

Deployment Discovery Service: A REST service which encapsulates the functionality of the Deployment Discovery Engine in the form of a REST API method. Internally, this service triggers the execution of the Deployment Discovery Orchestrator upon each service request.
Deployment Discovery Orchestrator: it takes care of orchestrating the execution of the rest of the components in the architecture in order to support the satisfaction of the best deployment discovery request. For each such request, this component invokes the Semantic Information Extractor to retrieve the most suitable information for the analysis from the Semantic KB which is then inserted in the form of facts inside the Knowledge Base. Then, the all rules of that Knowledge Base are fired which might invoke both the Semantic Information Extractor for the retrieval of additional information as well as the Smart Service Discovery & Selection Tool for inferring equivalence facts between different BPaaS elements.
Semantic Information Extractor: this component is capable of forming SPARQL queries which are then issued over the Semantic KB. It is also capable of transforming the query results into domain objects that can then be inserted inside the Knowledge Base.
Knowledge Base/KB: this component represents a normal knowledge base in which the rules for the best BPaaS deployment discovery are stored. This KB also enables the generation of sessions via which facts can be stored and rules can be fired to operate over these facts.
Smart Service Discovery & Selection Tool: this component has been produced in the research WP of the CloudSocket project ( provide link to the innovation shop). It enables the semantic functional and non-functional discovery of various kinds of services as well as the global selection of services in the context of a specific BPaaS workflow. It is mainly used in the Deployment Discovery Engine in order to infer the equivalence between services ( check if this is true or it can be considered as a future feature to implement.

Architecture of the Deployment Discovery Engine.

Installation Manual

The component has been implemented purely in java. It depends on other external components / frameworks like Spring, swagger, jersey and sesame/rdf4j. The following figure explicates the sole artefact generated mapping to this component as well as its dependencies on other artefacts (at a coarse-grained level to simplify the figure).

Artifact dependencies for the Deployment Discovery Engine.

Development

Currently, the following requirements hold for this component:

Oracle's 1.7.x JDK or higher
Apache tomcat 1.7 or higher
Maven tool for code compilation and packaging

The installation procedure to be followed is the one given below:

Download source code from https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/evaluation_dd/repository/archive.tar?ref=master
Unzip code with tar (or any other tool)
Go to the root directory of the installed code
Change the configuration file to provide the right access information for the Virtuoso Triple Store (https://virtuoso.openlinksw.com/)
Run:
```
 mvn clean install 
```
and then:
```
 mvn war:war 
```
Move the war file to the webapps directory of tomcat and start tomcat, if not yet started
Test installation by entering in your browser the following URL: http://localhost:8080/evaluation-dd/

Production

The same instructions as above hold for this component

Test Cases

You can run test cases directly from the URL (http://localhost:8080/evaluation-dd in your development environment) of the REST service due to the use of swagger (http://swagger.io) which enables the execution of the API method exposed by exploiting user input provided in a form-based manner. The following test cases are envisioned:

Retrieve the best deployments for a BPaaS

This test case attempts to assess whether the core functionality of the Deployment Discovery Engine is correctly delivered. In this respect, the broker should guarantee that there exist deployments for the required BPaaS to be analysed or for BPaaSes that are similar or equivalent to it and have been coupled with monitoring information (which could be done by exploring the content of the Semantic KB). Once this check is done, then the respective testing of the best deployment discovery functionality can proceed according to the following user story (where the id of the BPaaS can be of course interchanged with the one for which the analysis can be really performed).

Suppose that the user desires to find the best deployments of the "SendInvoiceSaaSEurope" BPaaS. Then, he/she can browse the methods of the API and click on the one named as "deploymentDiscovery". The following screenshot indicates the upper part of the description of the method when selected:

Swagger overview of the "deploymentDiscovery" method of the Deployment Discovery Service.

Next, he/she can fill in the required input parameters, i.e., the name of the broker ("bwcon"), the id of the BPaaS bundle ("SendInvoiceSaaSEurope") and optionally the analysis period. As he/she does require the whole execution history to be considered, the input to the "analysisPeriod" parameter is not given. Once the filling of the parameters is completed, then he/she can press the "Try it out!" button.

Finally, he/she will be able to see the respective curl command issued, the request URL, the corresponding results, as well as the response status and headers as indicated in the following figure:

Result of calling the deploymentDiscovery method of the Deployment Discovery Service.

User Manual

API Specification

The reader should refer to the Swagger-based API web page (http://localhost:8080/evaluation-dd for a local installation or http://134.60.64.222:8080/evaluation-dd for an existing remote installation) for browsing the API on-line documentation with the capability to execute the API methods. In the following, the API methods of the Process Mining Service are analysed below:

deploymentDiscovery

This method enables to execute the best deployment discovery functionality and retrieve back the corresponding results:

 POST dd/deploymentDiscovery/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the best deployment discovery needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the analysis needs to be performed (obligatory)
analysisPeriod: denotes the extent of the execution history from the current moment in the form of a Java period (a possible value, e.g., "P1M" would then declare a period of 1 month until now) (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output will be in either JSON or XML form for now. A sample of a JSON-based output is given below:

 <b> to include the description of the result here </b>

Handbook

Semantic Knowledge Base 2.3

2017-05-25T13:11:45Z

1 Summary
2 Component responsible
3 Architecture Design
4 Installation Manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 API Specification
- 6.2 Handbook

Summary

The Semantic Knowledge Base (Semantic KB) is a composite component in the BPaaS Evaluation Environment which is responsible for the storage and management of the semantic information collected by the Harvesting Engine. It represents a dependency point for most of the other components in the evaluation environment as all of these components need to interact with it in order to retrieve or store semantic information. The only components currently independent from the Semantic KB are those at the upper level of the evaluation environment architecture, i.e., the Hybrid Business Dashboard and the Hybrid Business Management Tool. This is due to the fact that the Semantic KB is not directly involved in the execution of a certain BPaaS analysis functionality.

This component has been realised in the form of a REST API which lies on top of an existing Triple Store named as Virtuoso. This API offers typical semantic data management functionalities, such as querying, importing, exporting, insertion, updating, deletion. In addition, the use of a Triple Store enables the exploration of other facilities that can be offered that are more triple-store-specific. However, this is not actually required as the REST API should be used instead which enables decoupling from any technical peculiarities of the corresponding triple store exploited thus enabling the future interchange of that triple store implementation with another one.

The main functionalities offered by the REST API can be summarised as follows:

SPARQL query issuing -- main functionality that has been widely used by all analysis engines
semantic information importing -- various formats are supported for this but Turtle is the one mostly recommended for performance reasons
semantic information insertion/updating/deletion -- triples can be inserted, inserted or deleted at any time point by exploiting the Sesame/rdf4j interface bridge of Virtuoso via issuing SPARUL statements
semantic information exporting -- semantic information exporting is also supported in different forms - similarly to the case of importing, Turtle is recommended also for file size reasons

It should be noted that for administration purposes, Virtuoso also offers the Virtuoso Conductor which is a web interface that enables to manage in a web-based manner your triple store implementation. This allows for the creation and deletion of whole graphs, the querying of the semantic information, the importing of such information and its updating. As such, it can be a complement to the API offered which can enable an administrator to intervene at any time point and browse, inspect and adjust the content of the Triple Store, if needed.

This component can be easily replaced by another component developed externally with respect to the CloudSocket organisation. The only requirement for such a replacement is that the same REST interface should be realised as it is currently exploited by many other components in the BPaaS Evaluation Environment architecture (including the Harvesting Engine, the Process Mining Engine, the Deployment Discovery Engine and the Conceptual Analytics Engine).

The following table indicates the details of the component.

Type of ownership	Update
Original tool	Virtuoso Triple Store
Planned OS license	Mozilla Public Licence 2.0.
Reference community	ADOxx Community.

Consist of

Semantic KB Service (REST API exposing the core functionality of the Semantic KB)

Depends on

This component is entirely independent of the others in the BPaaS Evaluation Environment Architecture

Component responsible

Developer	Email	Company
Kyriakos Kritikos	kritikos@ics.forth.gr	FORTH

Architecture Design

The Semantic KB is coloured in orange in the overall BPaaS Evaluation Environment architecture depicted in the figure below. As it can be seen, it is invoked by all the engines involved in this environment.

The final, internal architecture of the Semantic KB, which is depicted below, comprises the following 2 main components:

Semantic KB Service: A REST service which encapsulates the semantic data management functionality exposed by the Semantic KB in the form of a set of REST API methods.
Triple Store: an internal triple store which enables the semantic data management functionality by providing the right facilities for its realisation. This maps to using the Sesame interface bridge in order to be able to exploit Sesame as the implementation of the Semantic KB Service which can accordingly connect to the triple store in order to execute the semantic data management functionality requested.

Architecture of the Semantic KB.

Installation Manual

The component has been implemented purely in java. It depends on other external components/frameworks like Spring, swagger, jersey and sesame/rdf4j. The following figure explicates the sole artefact generated mapping to this component as well as its dependencies on other artefacts (at a coarse-grained level to simplify the figure).

Artifact dependencies for the Semantic KB.

Development

Currently, the following requirements hold for this component:

Oracle's 1.7.x JDK or higher
Apache tomcat 1.7 or higher
Maven tool for code compilation and packaging

The installation procedure to be followed is the one given below:

Download source code from https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/evaluation_skb/repository/archive.tar?ref=master (to be modified)
Unzip code with tar (or any other tool)
Go to the root directory of the installed code
Change the configuration file to provide the right access information for the Virtuoso Triple Store (https://virtuoso.openlinksw.com/)
Run:
```
 mvn clean install 
```
and then:
```
 mvn war:war 
```
Move the war file to the webapps directory of tomcat and start tomcat, if not yet started
Test installation by entering in your browser the following URL: http://localhost:8080/skb/ (eventual endpoint)

Production

The same instructions as above hold for this component

Test Cases

You can run test cases directly from the URL (http://localhost:8080/skb in your development environment) of the REST service due to the use of swagger (http://swagger.io) which enables the execution of the API method exposed by exploiting user input provided in a form-based manner.

The loading of the API can unveil that it is up and running but its correct functioning is the one that needs to be further tested. However, this correct functioning depends on the proper functioning of the Virtuoso Triple Store. In this sense, the tester should also attempt to connect to the virtuoso conductor by supplying the corresponding URL (http://localost:8890 for a local installation or http://http://134.60.64.222:8890 for an existing remote installation). If this is successful, he/she can also check whether some functionality can be executed by, e.g., executing a simple SPARQL query. One all these steps are performed, the actual testing of the API can proceed.

The following two test cases are envisioned which map to user stories that actually involve a semantic/ontology expert (acting also as an administrator of the Semantic KB). Such a user is required to be involved here due to the respective expertise and knowledge needed for correctly carrying out the testing cases. In principle, the broker should have employed such a user in order to take care the administration of the underlying triple store.

Linked Data Querying

This test case attempts to check whether the querying functionality of the API operates normally and returns correct results. Such results can also be compared to those returned by the native or the conductor interface of Virtuoso. Please note for such a comparison of the ordering can be different unless there are some elements in the corresponding query that impose a certain order of results presentation that has to be enforced. Please also take into account that this type of testing can also be used for checking if the harvesting and linking of semantic information is correct.

By relying on the latter observation, this test case attempts to run a specific test query of the Harvesting Engine in order to check the correctness of the harvesting and semantic linking. Thus, suppose that the user desires to run the first test query which attempts to retrieve all BPaaS workflow tasks and their input and output parameters. He/she can then browse the methods of the API and click on the one named as "ldQuery". The following screenshot indicates the upper part of the description of the method when selected:

Swagger overview of the "ldQuery" method of the Semantic KB Service.

Next, he/she can fill in the required input parameters, i.e., the URI of the graph mapping to the broker-specific fragment, the SPARQL (test) query itself, the timeout value and the max number of rows to be returned. As the user is not impatient, he/she does not provide a value for the "timeout" parameter and desires to obtain all results so lets the default value of the "maxRows" input parameter to apply. Once the filling of the parameters is completed, then he/she can press the "Try it out!" button.

Finally, he/she will be able to see the respective curl command issued, the request URL, the corresponding results, as well as the response status and headers as indicated in the following figure:

Result of calling the ldQuery method of the Semantic KB Service.

Graph Deletion

This test case attempts to check whether the updating functionality of the Semantic KB Service works correctly. It maps to the usual situation within a deployment environment that the triple store administrator desires to remove the broker-specific graphs created to initiate again the harvesting and linking of deployment, monitoring and registry information after a bug fix. Such a task can be of course executed via the Virtuoso Conductor as it enables the user-intuitive browsing and management of all the graphs created but here we need to focus on the testing of the Semantic KB Service. This means that an SPARUL statement needs to be executed via the "ldUpdate" method of this service in order to delete these graphs. The following user story applies to this test case.

Suppose that the user desires to delete the graphs of the sole broker currently serviced namely "bwcon". To this end, he/she will first browse the methods of the API and click on the one named as "ldUpdate". The following screenshot indicates the upper part of the description of the method when selected:

Swagger overview of the "ldUpdate" method of the Semantic KB Service.

Next, he/she can fill in the required input parameters. In this case, only the SPARUL statement needs to be provided for the first parameter, the "update" one as the rest of the parameters do not need to be filled in with the value of "DROP <http://www.cloudsocket.eu/evaluation/bwcon>; DROP <http://www.cloudsocket.eu/evaluation/bwcon/kpis>; DROP <http://www.cloudsocket.eu/evaluation/bwcon/history>" (these are the graphs mapping to the "bwcon" broker). Once the filling of the parameters is completed, then he/she can press the "Try it out!" button.

Finally, he/she will be able to see the respective curl command issued, the request URL, the corresponding results, as well as the response status and headers as indicated in the following figure:

Result of calling the ldQuery method of the Semantic KB Service.

User Manual

API Specification

The reader should refer to the Swagger-based API web page (http://localhost:8080/skb for a local installation or http://134.60.64.222:8080/skb for an existing remote installation) for browsing the API on-line documentation with the capability to execute the API methods. In the following, the API methods of the Process Mining Service are analysed below:

ldQuery

This method enables to run a SPARQL query over the underlying Triple Store:

 POST skb/ldQuery

The expected input format is multipart/form-data. The following input parameters are expected:

query: denotes the SPARQL query to be issued (obligatory)
graphURI: denotes the URI of the graph to be queried. This parameter is optional as the URI of the graph can be provided in the FROM clause of the query (optional)
timeout: denotes the maximum amount of time to wait until the query results can be received. It defaults to the value of 0 which means unlimited (optional)
maxRows: denotes the maximum number of query results to be returned. It defaults to the value of 0 which means all. This parameter is ignored if the query has the LIMIT SPARQL clause involved in it. This means that the query semantics is given priority over the semantics of this parameter (obligatory)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
401 -- Access denied exception
404 -- The resource requested was not found
406 -- The format requested for the method output is not supported

The output will be in various RDF forms ("application/sparql-results+xml", "application/sparql-results+json", "text/csv" and "text/tab-separated-values"). A sample of the JSON-based form for an output is given below:

 { "head": { "link": [], "vars": ["bpaas", "id"] }, "results": { "distinct": false, "ordered": true, "bindings": [ { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceSaaSEurope" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceSaaSEurope" }}, { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceSaaSWorldwide" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceSaaSWorldwide" }}, { "bpaas": { "type": "uri", "value": "http://www.cloudsocket.eu/evaluation#BPaaS_bwcon_SendInvoiceIaaSEurope" } , "id": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#string", "value": "SendInvoiceIaaSEurope" }} ] } }

ldExport

This method enables to export the contents of a graph or part of these contents in various RDF formats (RDF/XML, Turtle, CSV, etc.):

 POST skb/ldExport

The expected input format is multipart/form-data. The following input parameters are expected:

graphURI: denotes the URI of the graph whose content should be exported. If this URI is not given, then the default graph in the Triple Store is assumed (optional)
subjURI: denotes the URI of the subject resource to be exported. If this URI is given, then only statements containing this URI as subject are exported (optional)
predURI: denotes the URI of the predicate to be exported. If this URI is given, then only statements containing this predicate are exported (optional)
objURI: denotes the URI of the object resource to be exported. If this URI is given, then only statements containing this URI as object are exported (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
401 -- Access denied exception
404 -- The resource requested was not found
406 -- The format requested for the method output is not supported
500 -- An internal server error has occurred preventing the service from appropriately serving the request

The output will be in various RDF forms (RDF/XML, Turtle, NTriples, CSV, etc.). A sample of the Turtle form for an output is given below (produced by requesting to obtain only statements involving the <http://www.cloudsocket.eu/evaluation#allocation predicate for the bwcon broker):

 @prefix : <http://www.cloudsocket.eu/evaluation#> . :BPaaS_SendInvoice :allocation :Workflow_SendInvoice_Allocation2, :Workflow_SendInvoice_Allocation1. :BPaaS_ChristmasSending :allocation :Workflow_ChristmasSending_Allocation2, :Workflow_ChristmasSending_Allocation1.

ldImport

This method enables to import a linked data file which can be either put inside the request body or can be downloaded from a specific URL given as input:

 POST skb/ldImport

The expected input format is either "application/rdf+xml" or "text/rdf+n3" or "text/turtle" for the inline request while just query parameters are involved in the main body. The following input parameters are expected:

url: denotes the URL from which the linked data content can be downloaded. It is optional as there is also the option to provide this content inline in the request (optional)
uri: denotes the URI of the graph that will be mapped to the imported content. Absence of this parameter signifies that the imported content will be inserted in the default graph of the Triple Store (optional)
format: denotes the format of the content to be imported. Only required in case that this content is to be downloaded from the given URL (optional)
blocking: indicates whether the request should be blocked waiting for the importing to end or not. In the latter case, the method returns an id which can be used to query the status of the import by calling the "importStatus" method (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
401 -- Access denied exception
404 -- The resource requested was not found
406 -- The format requested for the method output is not supported
500 -- An internal server error has occurred preventing the service from appropriately serving the request
503 -- The service is not available

The output will be in either JSON or XML form. A sample of the JSON form for an output is given below (highlighting that a non-blocking request was issued for which the importing is currently running):

 { "importID": "100", "status": "RUNNING" }

importStatus

This method enables to obtain the status of an ongoing RDF import request:

 POST skb/importStatus

The expected input format is multipart/form-data. The following input parameters are expected:

importId: denotes the importId of an ongoing import request which should have been returned when the "ldImport" method was called in a non-blocking way (obligatory)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
401 -- Access denied exception
404 -- The resource requested was not found
500 -- An internal server error has occurred preventing the service from appropriately serving the request

The output will be in either JSON or XML form. A sample of the JSON form for an output is given below (actually almost the same as in the case of the "ldImport" method but now denoting a successful result):

 { "importID": "100", "status": "FINISHED" }

ldUpdate

This method enables to update the content of an existing graph in the Triple Store by issuing a SPARUL statement:

 POST skb/ldUpdate

The expected input format is multipart/form-data. The following input parameters are expected:

update: denotes the update statement in SPARUL (obligatory)
baseURI: denotes the base URI via which relative URI statements in the SPARUL query can be resolved (optional)
timeout: denotes the maximum amount of time that the broker can wait until the update is performed (optional)
maxRows: indicates the maximum number of rows that can be affected from the execution of the SPARUL statement (optional)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
401 -- Access denied exception
500 -- An internal server error has occurred preventing the service from appropriately serving the request

A respective message is given in the response which indicates the successful outcome of the execution of the update statement:

 Update was performed successfully

Handbook

Process Mining Engine 2.9

2017-05-25T09:12:16Z

1 Summary
2 Component responsible
3 Architecture Design
4 Installation Manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 API Specification
  - 6.1.1 pmAlgorithms
  - 6.1.2 mine
- 6.2 Handbook

Summary

The Process Mining Engine is a composite component in the BPaaS Evaluation Environment which is responsible for executing process mining algorithms over the execution history of a BPaaS stored in the Semantic KB. This component is invoked by the Hybrid Dashboard to execute this required type of BPaaS analysis and visualise the respective analysis results according to the best possible visualisation metaphor.

While various kinds of process mining algorithms exist, we have selected to re-use only one kind for the moment. This kind maps to the recreation of a process model from the process log. This kind is quite useful for the optimisation of the BPaaS modelling as it enables to check which paths in the BPaaS workflow model are frequently executed, which are scarcely executed and which are not at all. As such, this can signify an indication to the modeller that possibly the paths that are never followed do not need to be modelled anymore. Concerning the other kinds of process mining algorithms that we intended to support, which include decision and organisational mining, they were not integrated for some technical reasons mapping mainly to the lack of some required data that could be collected from particular information sources in the CloudSocket platform (see D3.6 for more details). On the other hand, we do support semantic process mining as we can exploit semantic annotations over tasks in order to improve the accuracy of the process mining algorithms currently exploited.

The process model re-creation algorithms that we exploit come from the ProM process mining framework and constitute state-of-the-art algorithms. They are exploited as they are by appropriately creating a log from the Semantic KB mapping to the execution history of a certain BPaaS (either solely coupled to a specific customer or across all customers of that BPaaS) and then passing that log as input to these algorithms. This log conforms to the XES standard. Depending on the broker choice, this log can be formed by including either the names of the BPaaS workflow tasks in the log entries or the names of the concepts which have been used to annotate these workflow tasks. Upon successful execution, the result of each algorithm is then transformed again by exploiting the facilities of the ProM framework into a file conforming to BPMN which is the de facto standard adopted by CloudSocket.

As enabled as a feature by the Harvesting Engine, the Process Mining Engine does support multi-tenancy as it allows each broker to focus the analysis only on its own fragment in the Semantic KB. The whole process mining functionality is encapsulated for that reason in the form of a REST API which does take as input the id of the broker for which the analysis has to be performed. Apart from the core process mining functionality, this REST API offers a utility function that lists all state-of-the-art process mining algorithms that can be exploited by the broker. In this sense, via the interaction with the Hybrid Business Dashboard, the broker is listed with all these algorithms, he/she can select the one that is desired by him/her and then initiate its execution.

The Process Mining Engine can be easily replaced by another component developed externally by an organisation outside the CloudSocket consortium. This is due to the fact that this component is not tightly coupled with any other internal component in the BPaaS Evaluation Environment. It just communicates with the Semantic KB in order to retrieve the execution history of a certain BPaaS and exposes an API to the Hybrid Business Dashboard in order to enable its functionality to be invoked. In this sense, any replacement component would just have to conform to that API as well as be able to draw the correct information from the content of the Semantic KB.

The following table indicates the details of the component.

Type of ownership	Creation
Original tool	ProM Process Mining Framework
Planned OS license	Mozilla Public Licence 2.0.
Reference community	ADOxx Community.

Consist of

Process Mining Service (REST API exposing the core functionality of this engine)

Depends on

Semantic KB

Component responsible

Developer	Email	Company
Kyriakos Kritikos	kritikos@ics.forth.gr	FORTH

Architecture Design

The Process Mining Engine is is coloured in blue in the overall BPaaS Evaluation Environment architecture depicted in the figure below. As it can be seen, it is invoked by the Business Management Tool in order to invoke the right state-of-the-art process mining algorithm. This invocation, in turn, leads to exploiting the content of the Semantic KB at the Data Layer in order to support the creation of the log to be mined.

The final, internal architecture of the Process Mining Engine, which is depicted below, comprises the following 5 main components:

Process Mining Service: A REST service which encapsulates the whole functionality of the Process Mining Engine in the form of REST API methods. Each method invoked triggers the execution of the Process Mining Orchestrator.
Process Mining Orchestrator takes care of orchestrating the execution of the rest of the components in the architecture in order to support the satisfaction of the process mining request. For each process mining request, this component first invokes the Query Creator to create a SPARQL query which is then issued over the Semantic KB. The query result is then used to invoke the Log Creator in order to produce the required log which is exploited to execute the process mining algorithm selected from the Process Mining Library. The mining result is then transformed into BPMN form by exploiting a certain transformation algorithm from the Process Mining Library depending on the original form of the output supported by the executed process mining algorithm.
Query Creator: this component, depending on the broker input which designates the BPaaS to be mined and optionally: (a) the customer to which the analysis should be restrained, (b) the period of the execution history and (c) whether semantic mining is to be performed, dynamically creates the right SPARQL query which can be issued over the Semantic KB in order to accurately obtain the right process log-oriented information to be used for the mining.
Log Creator: this component takes the result of the SPARQL query execution and creates a log out of it which conforms to the XES standard. Depending on whether normal or semantic process mining needs to be supported, either the BPaaS workflow task names or their annotations are used to populate each entry of the log.
Process Mining Library: this library has been constructed by exploiting some state-of-the-art process mining algorithms from ProM as well as the needed transformation algorithms (e.g., from petri net to BPMN) from the same framework that guarantee the final production of a BPMN recreated process model.

Architecture of the Process Mining Engine.

Installation Manual

The component has been implemented purely in java. It depends on other external components / frameworks like ProM, OpenXES, Spring, swagger, jersey and sesame/open rdf. The following figure explicates the sole artefact generated mapping to this component as well as its dependencies on other artefacts (at a coarse-grained level to simplify the figure).

Artifact dependencies for the Process Mining Engine.

Development

Currently, the following requirements hold for this component:

Oracle's 1.7.x JDK or higher
Apache tomcat 1.7 or higher
Maven tool for code compilation and packaging

The installation procedure to be followed is the one given below:

Download source code from https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/evaluation_pm/repository/archive.tar?ref=master
Unzip code with tar (or any other tool)
Go to the root directory of the installed code
Change the configuration file to provide the right access information for the Virtuoso Triple Store (https://virtuoso.openlinksw.com/)
Run:
```
 mvn clean install 
```
and then:
```
 mvn war:war 
```
Move the war file to the webapps directory of tomcat and start tomcat, if not yet started
Test installation by entering in your browser the following URL: http://localhost:8080/evaluation-pm/

Production

The same instructions as above hold for this component

Test Cases

You can run test cases directly from the URL (http://localhost:8080/evaluation-pm in your development environment) of the REST service due to the use of swagger (http://swagger.io) which enables the execution of the API method exposed by exploiting user input provided in a form-based manner. The following test cases are envisioned:

Obtain all state-of-the-art process mining algorithms supported

At first contact, a broker might desire to see which process mining algorithms can be exploited. Moreover, this can be a first type of basic testing that the service is not only up but is also running correctly. To this end, the broker will go to the endpoint URL of the Process Mining Service and will click to expand the pmAlgorithms method. Then he/she can just press the "Try it out!" button to see the corresponding result. The following screenshot shows both the swagger-based description of this API method and the corresponding result of its invocation.

Result of executing the pmAlgorithms method of the Process Mining Service.

Run a process mining algorithm

This test case attempts to assess whether the core functionality of the Process Mining Engine is correctly delivered. In this respect, the broker should guarantee that the execution history of the BPaS to be examined already exists and has been stored in the Semantic KB. Once this check is done, then the respective testing of the mining functionality can proceed according to the following user story (where the id of the BPaaS can be of course interchanged with the one for which the execution history does exist).

Suppose that the user desires to mine the execution history of the "SendInvoiceSaaSEurope" BPaaS. Then, he/she can browse the methods of the API and click on the one named as "mine". The following screenshot indicates the upper part of the description of the method when selected:

Swagger overview of the mine method of the Process Mining Service.

Next, he/she can fill in the required input parameters, i.e., the name of the broker ("bwcon"), the id of the BPaaS bundle ("SendInvoiceSaaSEurope"). As he/she does not need to restrain the mining over one tenant as well as requires the whole execution history to be considered, the rest of the parameters are not touched. One this is done, then he/she can press the "Try it out!" button.

Finally, he/she will be able to see the respective curl command issued, the request URL, the corresponding results, as well as the response status and headers as indicated in the following figure:

Result of calling the mine method of the Process Mining Service.

User Manual

API Specification

The reader should refer to the Swagger-based API web page (http://localhost:8080/evaluation-pm for a local installation or http://134.60.64.222:8080/evaluation-pm for an existing remote installation) for browsing the API on-line documentation with the capability to execute the API methods. In the following, the API methods of the Process Mining Service are analysed below:

pmAlgorithms

This method enables to obtain the list of the state-of-the-art process mining algoriths that can be executed by the Process Mining Service. No input is required for executing this method.

 POST pm/pmAlgorithms

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
406 -- The format requested for the method output is not supported
500 -- An internal server error has been generated

The output can be in both json and xml formats. A sample for a json output is given below:

 [ "HeuristicsMiner", "HeuristicsCnetMiner", "ILPMiner", "InductiveMiner" ]

mine

This method enables to execute a certain state-of-the-art process mining algorithm (from those supported -- see previous method) over the execution history of a specific BPaaS:

 POST pm/mine/{brokerId}

The expected input format is multipart/form-data. The following input parameters are expected:

brokerId: denotes the id of the broker for which the mining needs to be performed. It is a path parameter as it can be seen from the method relative URL (obligatory)
bundleId: denotes the id of the BPaaS bundle for which the mining needs to be performed (obligatory)
minePeriod: denotes the extent of the execution history from the current moment in the form of a Java period (a possible value, e.g., "P1M" would then declare a period of 1 month until now) (optional)
tenant: denotes the id of the tenant/customer for which the mining can be performed. This will restrict the execution history to include only information pertaining to the instance(s) of the BPaaS at hand that have been purchased by this tenant (optional)
algorithm: denotes the name of the state-of-the-art algorithm to be used for the process mining. This name can be obtained by calling the previous method ("pmAlgorithms") (obligatory)
semantic: denotes whether semantic or normal process mining must be performed. The default value is false (favouring normal process mining) (obligatory)

The following response codes can be returned depending on whether the API method execution was successful or not:

200 -- The method execution was successful
400 -- Wrong parameter values have been provided
404 -- The resource requested (i.e., the broker) was not found
406 -- The format requested for the method output is not supported

The output will be strictly for now in BPMN form. A sample of such an output is given below:

 <b> to include the description of the BPMN process model here </b>

Handbook

Monitoring Engine (InfluxDB Prototype) 1.0

2017-05-22T08:16:44Z

TODO: add brief description according the D3.6

Developer manual 1.4

2017-04-21T08:55:04Z

1 Configuration of development environment
2 Specific fields

Configuration of development environment

To be completed

Specific fields

How to create specific fields

When a new form field is needed to be included in a user task, and it is not available. You, as a developer, need to follow these steps in order to create it:

Create the specific classes for this field.
Include the classes in the Spring configuration to be injected automatically when the WFE generate the form on the fly.
Include the attributes of the beans into the Spring configuration to be recognized by the renderer.
Add the new type field in the BPMN editor.
Include the specific styles in the css file.
Create simple workflow to see the new field

The following sections describe these steps through the message text field example, which shows a non modifiable message box for the user.

Create the specific classes for this field

Create the custom component to be included in the custom field. You can include here what you consider, since it will be rendered on the fly in the associated user task.

 package net.atos.yourbpm.form; import com.vaadin.ui.*; public class MessageTextCustomComponent extends CustomComponent { private static final long serialVersionUID = 1L; private Label message; private Label label; MessageTextCustomComponent() { messageTextCustomComponentBuild("Message", ""); } MessageTextCustomComponent(String vLabel, String vMessage) { messageTextCustomComponentBuild(vLabel, vMessage); } private void messageTextCustomComponentBuild(String vLabel, String vMessage){ VerticalLayout layoutV = new VerticalLayout(); layoutV.setStyleName("messageTextField"); label = new Label(vLabel); label.setContentMode(Label.CONTENT_XHTML); label.setStyleName("messageTextField_title"); message = new Label(vMessage); message.setContentMode(Label.CONTENT_XHTML); message.setStyleName("messageTextField_message"); layoutV.addComponent(label); layoutV.addComponent(message); setCompositionRoot(layoutV); } public String getLabel() { return (String) label.getValue(); } public void setLabel(String value) { label.setValue(value);; } public String getMessage() { return (String) message.getValue(); } public void setMessage(String vMessage) { message.setValue(vMessage); } }

NOTE: remember to set the composition of the root through the method "setCompositionRoot(yourComposition)". If you don't do it, you will receive and error indicating that the root is null when the engine is rendering the page in execution time (but for building time is correct).

Create the custom field to be included into the form associated to the user task.

 package net.atos.yourbpm.form; import org.vaadin.addon.customfield.CustomField; public class MessageTextField extends CustomField { private static final long serialVersionUID = 1L; private MessageTextCustomComponent messageTextCustomComponent; public MessageTextField() { messageTextCustomComponent = new MessageTextCustomComponent(); setCompositionRoot(messageTextCustomComponent); } public MessageTextField(String head, String message) { messageTextCustomComponent = new MessageTextCustomComponent(head, message); setCompositionRoot(messageTextCustomComponent); } @Override public Class<?> getType() { return String.class; } @Override public Object getValue() { return messageTextCustomComponent.getMessage(); } @Override public void setValue(Object value){ messageTextCustomComponent.setMessage((String) value); } public void setLabel(Object value){ messageTextCustomComponent.setLabel((String) value); } }

You have to include the MessageTextCustomComponent into your custom field as an attribute of the new field. Remember to include the new custom field to the root as you have done for the custom component (method "setCompositionRoot(messageTextCustomComponent)"). The engine will render all the components and it will need to have the main root (if not, an execution error will appear indicating that the root is null).

You also need to override the abstract methods of the CustomField Class:

- public Class<?> getType()
- public Object getValue()
- public void setValue(Object value)

Create the FormType, which is responsible to convert the value between the new field type and the Model:

 package net.atos.yourbpm.form; import org.activiti.engine.form.AbstractFormType; public class MessageTextFormType extends AbstractFormType { private static final long serialVersionUID = 1L; public static final String TYPE_NAME = "messageText"; public String getName() { return TYPE_NAME; } public Object convertFormValueToModelValue(String propertyValue) { return propertyValue; } public String convertModelValueToFormValue(Object modelValue) { if (modelValue == null) { return null; } return modelValue.toString(); } }

Create the property renderer, which is responsible to include the new field into the user task:

 package net.atos.yourbpm.form; import org.activiti.engine.form.FormProperty; import org.activiti.explorer.ui.form.AbstractFormPropertyRenderer; import com.vaadin.ui.Field; public class MessageTextPropertyRenderer extends AbstractFormPropertyRenderer { private static final long serialVersionUID = 1L; public MessageTextPropertyRenderer() { super(MessageTextFormType.class); } public Field getPropertyField(FormProperty formProperty) { MessageTextField messagetextField = new MessageTextField(formProperty.getName(), formProperty.getValue()); return messagetextField; } }

Include the attributes of the beans into the Spring configuration to be recognized by the renderer.

Include the new field type into the activiti-custom-context.xml for the bean "processEngineConfiguration"
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration"> <property name="dataSource" ref="dataSource" /> <property name="transactionManager" ref="transactionManager" /> <property name="databaseSchemaUpdate" value="true" /> <property name="jobExecutorActivate" value="true" /> <property name="enableDatabaseEventLogging" value="true" /> <property name="customFormTypes"> <list> <bean class="org.activiti.explorer.form.UserFormType"/> <bean class="org.activiti.explorer.form.ProcessDefinitionFormType"/> <bean class="org.activiti.explorer.form.MonthFormType"/>  <bean class="net.atos.yourbpm.form.TextAreaFormType"/> <bean class="net.atos.yourbpm.form.RadioGroupFormType"/> <bean class="net.atos.yourbpm.form.UploadFormType"/> <bean class="net.atos.yourbpm.form.InvoiceItemsFormType"/> <bean class="net.atos.yourbpm.form.MessageTextFormType"/> </list> </property> </bean>

Include the new renderer into the activiti-ui-context.xml for the bean "formPropertyRendererManager"

 <!-- Custom form property renderers can be plugged in here --> <bean id="formPropertyRendererManager" class="org.activiti.explorer.ui.form.FormPropertyRendererManager" lazy-init="true"> <!-- Default renderer --> <property name="noTypePropertyRenderer"> <bean class="org.activiti.explorer.ui.form.StringFormPropertyRenderer" /> </property> <!-- Renderers by property type --> <property name="propertyRenderers"> <list> <bean class="org.activiti.explorer.ui.form.StringFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.EnumFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.LongFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.DoubleFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.DateFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.UserFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.BooleanFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.ProcessDefinitionFormPropertyRenderer" /> <bean class="org.activiti.explorer.ui.form.MonthFormPropertyRenderer" /> <!--YourBPM--> <bean class="net.atos.yourbpm.form.TextAreaFormPropertyRenderer" /> <bean class="net.atos.yourbpm.form.RadioGroupPropertyRenderer" /> <bean class="net.atos.yourbpm.form.UploadPropertyRenderer" /> <bean class="net.atos.yourbpm.form.InvoiceItemsPropertyRenderer" /> <bean class="net.atos.yourbpm.form.MessageTextPropertyRenderer" /> </list> </property> </bean>

NOTE: If you are using the activiti rest module, it is necessary to add the field type in the activiti-custom-context.xml, since the engine parsers all possible attributes of the BPMN 2.0 file, so although it is not responsible to render the graphical user interface, we need to declare it in order to allow to deploy the BPMN 2.0 files that include the new field type.

Add the new type field in the BPMN editor.

Now, you can used it in your BPMN2.0 files that you want to deploy. However we can include this new field in the user task during the design phase in order to simplify the creation of the new forms. Add the name "messageText" that you have associated in the Class MessageTextFormType --> "public static final String TYPE_NAME = "messageText";""

Add this "messageText" name into the html file --> /activiti-webapp-explorer2/src/main/webapp/editor-app/configuration/properties/form-properties-popup.html

 <div class="form-group"> <label for="typeField">{{'PROPERTY.FORMPROPERTIES.TYPE' | translate}}</label> <select id="typeField" class="form-control" ng-model="selectedProperties[0].type" ng-change="propertyTypeChanged()"> <option>string</option> <option>long</option> <option>boolean</option> <option>date</option> <option>enum</option> <!-YourBPM -> <option>textArea</option> <option>upload</option> <option>radioGroup</option> <option>invoiceItems</option> <option>messageText</option> </select> </div>

Include the specific styles in the css file.

Although the Vaadin framework abstract the GUI layer (html, javacript...), it is recommended not to include the styles into the own java classes for the different graphical user interface representation. Hence, in order to decouple the layers, you can associate different styles that you will be able to modify directly through the css file. For example, we have associated different styles for the message box:

Box of the message --> messageTextField
Title of the message --> messageTextField_title
Body of the message --> messageTextField_message

You only need to include in the style.css file (/activiti-webapp-explorer2/src/main/webapp/VAADIN/themes/activiti/styles.css) the styles that you want to applied for your application.

 .messageTextField { background-image: none; height: auto; width: 100%; background-color: #cae2f5; padding-top: 5px; padding-left: 10px; } .messageTextField_title { font-family: 'Open Sans Condensed', sans-serif; font-size: 15px; color: #333; font-weight: bold; } .messageTextField_message { color: #333; font-family: 'Muli', sans-serif; margin-bottom: 12px; }

Create simple workflow to see the new field

Enter in the design editor and create a simple workflow like this one:

Figure: simple workflow to test the specific field.

You can associate a field with the new type

Figure: assign the new field in the user task.

You will obtain the BPMN2.0 in order to deploy it:

 <?xml version='1.0' encoding='UTF-8'?> <definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:activiti="http://activiti.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.activiti.org/processdef"> <process id="Message_test" name="Message test" isExecutable="true"> <startEvent id="sid-CD793B83-B45D-45D2-81C4-B4A9D85FE77B"/> <userTask id="sid-3BC1FF99-E940-461F-8775-6DD41A21CDD6" name="Message Test" activiti:assignee="kermit"> <extensionElements> <activiti:formProperty id="messageError" name="Error Message" type="messageText" expression="The invoiceNinja service is not available" writable="false"/> </extensionElements> </userTask> <endEvent id="sid-A8AFB2A7-F8E5-449B-BDF8-22819398C337"/> <sequenceFlow id="sid-887B7D01-CB82-4596-8AAA-AEB247003E33" sourceRef="sid-CD793B83-B45D-45D2-81C4-B4A9D85FE77B" targetRef="sid-3BC1FF99-E940-461F-8775-6DD41A21CDD6"/> <sequenceFlow id="sid-2AF8F91C-C21C-46B5-BA5E-8AEB3D1CCFF3" sourceRef="sid-3BC1FF99-E940-461F-8775-6DD41A21CDD6" targetRef="sid-A8AFB2A7-F8E5-449B-BDF8-22819398C337"/> </process> <bpmndi:BPMNDiagram id="BPMNDiagram_test_para_Borrar"> <bpmndi:BPMNPlane bpmnElement="test_para_Borrar" id="BPMNPlane_test_para_Borrar"> <bpmndi:BPMNShape bpmnElement="sid-CD793B83-B45D-45D2-81C4-B4A9D85FE77B" id="BPMNShape_sid-CD793B83-B45D-45D2-81C4-B4A9D85FE77B"> <omgdc:Bounds height="30.0" width="30.0" x="84.99998474121094" y="167.99999237060547"/> </bpmndi:BPMNShape> <bpmndi:BPMNShape bpmnElement="sid-3BC1FF99-E940-461F-8775-6DD41A21CDD6" id="BPMNShape_sid-3BC1FF99-E940-461F-8775-6DD41A21CDD6"> <omgdc:Bounds height="80.0" width="100.0" x="300.0" y="142.99999237060547"/> </bpmndi:BPMNShape> <bpmndi:BPMNShape bpmnElement="sid-A8AFB2A7-F8E5-449B-BDF8-22819398C337" id="BPMNShape_sid-A8AFB2A7-F8E5-449B-BDF8-22819398C337"> <omgdc:Bounds height="28.0" width="28.0" x="590.9999847412109" y="168.99999237060547"/> </bpmndi:BPMNShape> <bpmndi:BPMNEdge bpmnElement="sid-2AF8F91C-C21C-46B5-BA5E-8AEB3D1CCFF3" id="BPMNEdge_sid-2AF8F91C-C21C-46B5-BA5E-8AEB3D1CCFF3"> <omgdi:waypoint x="400.0" y="182.99999237060547"/> <omgdi:waypoint x="590.9999847412109" y="182.99999237060547"/> </bpmndi:BPMNEdge> <bpmndi:BPMNEdge bpmnElement="sid-887B7D01-CB82-4596-8AAA-AEB247003E33" id="BPMNEdge_sid-887B7D01-CB82-4596-8AAA-AEB247003E33"> <omgdi:waypoint x="114.99998474121094" y="182.99999237060547"/> <omgdi:waypoint x="300.0" y="182.99999237060547"/> </bpmndi:BPMNEdge> </bpmndi:BPMNPlane> </bpmndi:BPMNDiagram> </definitions>

When you execute this workflow, the result will be a message where the Name is the Title and the value of the message can be assigned directly as a String or through a variable. It is not the objective of this section detailing how to use variables, hence the example use directly the expression string in the form.

Figure: Final result for the specific field.

How to introduce and modify basic fields

If you want to change the behaviour of an existing field cover by Vaadin, you don't need to create the custom field and component, since the field exists. Hence, you only need to jumps these steps, the rest of the steps are the same. For example you can review the TextArea and the RadioGroup fields included in the atos-form artifact.

Type of fields

The basic fields are included with the validations:

String Input
Numeric Input
Date Input
List of values

Figure: list of basic fields

YourBPM includes also the following fields to be included in the user tasks. All of these types are built following the previous steps (you can see them in the 'atos-forms' artifact):

Text area

Include the attribute <textarea> in the user task of your BPMN file. The following text area will appear in the form of your user task:

Figure: text area field

Upload file

Include the attribute <upload> in the user task of your BPMN file. The following upload component will appear in the form of your user task:

Figure: upload image field

RadioGroup

Include the attribute <radiogroup> in the user task of your BPMN file. The following radio-group component will appear in the form of your user task:

Figure: radio-group field

The input is managed through a json message, for example

 { "1":"Value 1", "2":"Value 2" }

The result will be the selected row in the radio group.

Invoice Input

Include the attribute <invoiceItems> in the user task of your BPMN file. The following radio-group component will appear in the form of your user task:

Figure: empty list of invoices

Figure: fill the invoice fields

Figure: inclusion of the new invoice row into the list

To populate the field you only need to create a json message, for example:

 [ { "id":0, "notes":"Product one", "cost":213.0, "qty":2,"product_key":"Product1" }, { "id":0, "notes":"Product two", "cost":234.0, "qty":4, "product_key":"Product 2" } ]

You can modify all the rows what you want and the result will be a json message with your updating data.

Workflow Engine Component 2.9

2017-04-20T13:43:38Z

1 Summary
2 Component responsible
3 Architecture design
4 Installation manual
- 4.1 Development
- 4.2 Production
5 Test Cases
6 User Manual
- 6.1 API specification
- 6.2 HandBook

Summary

It is responsible for managing the deployment, execution and management of the different workflow instances. It will be multi-tenant leading to executing a workflow instance on behalf of one organisation by also taking care of the corresponding workflow and organisation data level. This component will expose:

A graphical user interface for interacting with the different actors involved in the lifecycle of a workflow
A REST API interface allowing programmatic access to the different types of functionalities offered.

The main functionalities are:

Deploy/redeploy a workflow in the workflow-engine, instantiate it and execute it.
Manage and follow the workflow instances, according to the workflow description in BPMN.
Interact with manual tasks of the workflow.
Manage the workflow engine environment.

Moreover, the executable workflows are designed by the integrated Editor Workflow based on the abstract workflows. Their design has to be aligned with the workflow engine, since the standard BPMN2.0 doesn’t reach the complete execution definition and it is allowed to extend it in order to cover the whole execution through this workflow definition. As the deployment will be executed on the cloud, it is mandatory to define dynamically the self-contained executable workflows, which are included in the BPaaS bundle, following these standards.

The main roles to interact with the functionalities are:

CloudSocket Customer, who wants to use the purchased BPaaS bundles, which has been deployed automatically in the cloud. The customer can interact via the use of different roles: i) Knowledge Worker is responsible for managing the manual task of the workflows; ii) Process Responsible (technical skills) is in charge of dealing with the workflow engine, managing the workflow instances and following their status.
Broker, who is responsible for the configuration of the Workflow Engine, for performing tasks on behalf of customers (e.g., workflow instances actions), for inspecting/monitoring the status of all workflows deployed and especially the status of respective SLAs.
Platform operator, who may be the broker itself, and is responsible for checking if the Workflow Engine works as expected.

Additionally, there is a role that covers the functionalities for creating the executable workflow, which will be included in the BPaaS bundle, during the design phase. Therefore, this role is not directly related with the execution phase, but the internal component is completely integrated with the whole component.

Workflow Designer, which is responsible to design the executable workflows using the executable workflow editor in order to include the necessary information to transform the abstract workflows (independent of the workflow engine) to the executable workflows (associated to a specific workflow engine).

All the actors will interact directly with the graphical user interface and the system will manage automatically the authorization of the appropriated functionalities (see the user guide in the Manual section).

The following table indicates the details of the component.

Type of ownership	Extension
Original tool	FIWARE and Fed4FIRE project
Planned OS license	Apache License Version 2.0.
Reference community	Activiti community

Consist of

Web UI of workflow Engine (Graphical User interface)
Workflow Engine (core functionalities)

Depends on

Process Data Mediator Engine
IdM Marketplace

Component responsible

Developer	Email	Company
Roman Sosa Gonzalez	roman.sosa@atos.net	Atos
Joaquin Iranzo	Joaquin.iranzo@atos.net	Atos

Architecture design

Figure: General architecture for Workflow Engine

The general architecture of this component can be viewed by checking the components coloured in yellow, in above figure:

The below figure introduces in more detail the different modules of the Workflow Engine. The components are split into two main parts: i) one part is responsible for exposing the interfaces with the externals actors (other components, BPaaS Customers or Brokers); ii) the other is responsible for managing the business logic and the data layer.

The two layers allow decoupling the components between the presentation layer, which is directly related to the exposed interfaces (Graphical user interfaces and REST API) and the backend, which is responsible to provide all the functionalities and their persistence. Thus, the interface layer will focus on the look and feel or how to expose these interfaces; moreover it will manage the necessary calls to the backend layer, which will execute the actions.

Figure - Internal architecture for the Workflow Engine.

Front End layer:

REST Workflow module is responsible for exposing all the functionalities of the Workflow Engine through a REST API, allowing other components to interact with the engine programmatically, without human interaction. The main interactions are with the Cloud provider Engine, which is responsible to orchestrate the deployment of the BPaaS bundle in the cloud environment, and the Adaptation Engine, which is responsible to modify the environments and the currently running workflow instance in order to cover the new conditions.
Editor Workflow module is responsible for the editing of the executable workflows at the design phase. A graphical user interface is used for editing, modifying and generating the executable workflows, while the involved actors are the technical designer, who can be contracted by the brokers to take care of the workflow design task. However, the module cannot interact directly with the customers and their companies; it is thus exploitable only by brokers. Nevertheless, this editor is necessary to align the definition of the concrete workflows with the technology adopted in the workflow engine in order to deploy and execute them correctly in the cloud environment.
Explorer Workflow module is responsible for exposing a graphical user interface in order to interact with the human actors, such as brokers and customers. Hence, through this exposed dashboard, the different actors can manage the complete lifecycle of the deployed workflows, allowing interacting with the platform in an easy way, increasing the quality of experience (QoE).

Back End Layer

Core Workflow Engine module is responsible for managing all the functionalities of the Workflow Engine. Hence, the complexity of the business logic of all these functionalities is delegated to this module and an interface is exposed to interact with them. It is also responsible to interact with the data layer and to persist the workflow, its instances and the rest of the entities.
Worflow Parser module is responsible for managing the functionalities related to workflow parsing, such as the introduction of the real endpoints of the software components and the automatic generation of the service task tags to invoke WS and RESTFul services.
Bind Proxy is a small module to facilitate the interaction with the database. It is responsible for managing the binding between service tasks and the associated services. It has been introduced to avoid using the Core Workflow Engine module as a proxy, to only connect with the database in order to manage the binding actions. In this manner, using this module, the Workflow Parser is decoupled from the Core Workflow Engine.

Data layer

Workflow Data Base module is responsible for persisting all the data in order to support all the functionalities. The model definition is based on an entity-relationship schema to represent all the entities, including tenants, roles, workflows, instances, jobs… It exposes a standard interface to connect with the data base; nevertheless it is not its responsibility to create an Object-relational mapping to interact with the entities such as JPA in java or SQLAlchemy in Python, since that is delegated to the Core_Workflow_Engine and the Bind_Proxy to provide it.

All the components expose different types of interfaces in order to cover nonfunctional requirements, such as scalability, modularity and replaceability. Nevertheless, some of them are more coupled than others, since they work to provide features together.

The REST Workflow, Explorer Workflow and Core Workflow Engine components are working jointly in order to arrange the workflow execution. Therefore, it is possible to replace these components with implementations such as Camunda or BonitaSoft; and re-implementing the adaptation and extension developed in these components based on Activiti.
The Editor Workflow is partially independent of the base line workflow engine as its main duty is to create the executable workflow to be deployed in the specific engine, which might include concrete extensions on the standard BPMN. In this sense, this component could also be extended in order to work for other workflow engines. In addition, it could also be replaced by another workflow editing component which could be coupled to a specific workflow engine like Activiti or the one which totally replaces the agglomeration of the previously referred components in the previous bullet.
The Parser Worflow Manager and Bind Proxy have isolated functionalities and expose a Restful interface. Hence, it is feasible to substitute them by other components, if they maintain the exposed APIs.
Finally, the Workflow Engine use a persistence framework with support for custom SQL, stored procedures and advanced mappings, this allow to substitute the Workflow Data Base module by other relational databases as long as the new option follows the standard connectors, since this persistence framework creates an abstract layer between the entities and their persistence in the data base.

Installation manual

The component is developed in java based on Activiti 5.18; several frameworks are used such as Vaadin, Spring and mybatis. A MySQL database is used.

As a build automation tool, the components have used Maven, which describes how they are built, and their dependencies. This allows building and generating automatically the artifacts for the different environments (development and production) and their configuration for example with Eclipse. Besides, the use of this artifact will facilitate the continuous integration, which will be covered in Task T4.5 CloudSocket Integration and Consolidation.

The installation manual takes advantage of these tools, facilitating the installation at the different environments and describing the necessary steps to install the component and its modules. The following figure indicates the generated artefacts and their dependencies.

Figure - Artefacts of the Workflow Engine

Development

Requirements

Oracle JDK 1.7.x
Tomcat 7.x
Maven tool for building applications.

Installation Download source code. The workflow engine comprises two projects:

Workflow Parser: https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/workflow-parser.
- Components:
  - parser-service is the war package that exposes the API web (WS/REST)
  - parser-manager is responsible for managing the deployment and execution adaptations in order to obtain the appropiated BPMN2.0 file
  - BPMN20Manager is responsible for generating the automatic code, which will be included in the BPMN 2.0 file, to call WS. It will be integrated into the parser-manager component.
  - bpmn-model is the bpmn model that allows to manage and modify the BPM2.0 files following the standar.
  - parser-client is the proxy to call the interfaces, exposed in the parser-service component. This can be used by other applications or components in order to call the workflow parser.
Workflow Engine: https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/workflow-engine
- Components Activiti v5.18 (customized/modified):
  - activiti-webapp-rest2 is the war package that exposes the API REST of Activiti. It has been customized to cover the deployment and execution adaptations actions.
  - activiti-webapp-explorer2 is the war package that exposes the GUI of Activiti. It has been customized to manage the different functionalitis such as integration Idm, multi-tenant approach, introduction of tokens and groups....
  - activiti-rest is the component that execute the actions of the activiti-webapp-rest2. It has been modied to cover the actions of the activiti-webapp-rest2 component.
  - activiti-explorer is the component that execute the actions of the activiti-webapp-explorer2. It has been modied to cover the actions of the activiti-webapp-explorer2.
  - activiti-engine is the core of the engine. It has been modified to cover our functionalities.
- Rest of components:
  - atos-compel is responsible for discovering and binding the services (atomic services, software components...) with the service tasks included in the business process. It wraps the BPMN web editor and store the binding into the data base.
  - atos-bind is responsible for storing the binding and the tokens in the data base. it can be used by other components to access and manage the data of the system.
  - atos-service-tasks manages the different kind of service tasks such as the RESTproxy (which invokes remote services), monitoring service tasks (which allow to send monitoring data)
  - atos-form manages the different kind of fields that can be included in the user tasks in the business process.
  - demo-app is responsible for providing the different atomic services and software components, which have been used in the BPaaS bundles.

Create the necessary artifacs executing the following instruction for all of components:

 ( cd workflow-parser && mvn install ) ( cd workflow-engine && bin/bootstrap.sh && mvn install )

Copy the war file generated in the WorkflowEngineService, activiti-webapp-rest2 and emo-app modules into the webapps folder Tomcat.
```
 <modulePath>/target 
```

Development environment

Create the eclipse project executing the following instruction for all necessary components:
```
 mvn eclipse:eclipse 
```

Production

Requirements

OpenJDK 1.7.x
MySql 5.5.x
Tomcat 7.x
Nginx

Installation

Validate that the version of Java is the correct one, if not install the version
```
 #java -version java version "1.7.0_85" 
```

Install the requeriments

 sudo aptitude install mysql-server sudo aptitude install tomcat7 sudo aptitude install nginx

Configure the schema and user of the Activiti (the password of the root has been introduced during the MySql installation).

 mysql -h localhost -u root -p create database activiti_cloudsocket; create user cloudsocket@localhost identified by 'cloudsocket'; grant all privileges on activiti_cloudsocket.* to cloudsocket@localhost;

Configure the modules of Tomcat (if you want to change the default password). Modify the file "<activiti-webapp-rest2>/WEB-INF/classes/db.properties" and "<activiti-webapp-explorer2>/WEB-INF/classes/db.properties" in order to introduce the new password.
```
 db=mysql jdbc.driver=com.mysql.jdbc.Driver jdbc.url=jdbc:mysql://localhost:3306/activiti_cloudsocket jdbc.username=cloudsocket jdbc.password=cloudsocket 
```

Configure the Nginx to redirect the 80 --> 8080. Modify the file (/etc/nginx/sites-available)

 server { listen 0.0.0.0:80; server_name localhost 134.60.64.132; client_max_body_size 4G; access_log /var/log/nginx/workflow_Engine.access.log; error_log /var/log/nginx/workflow_Engine.error.log; location /demo-app-1.0/ { # an HTTP header important enough to have its own Wikipedia entry: # http://en.wikipedia.org/wiki/X-Forwarded-For proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # enable this if and only if you use HTTPS, this helps Rack # set the proper protocol for doing redirects: # proxy_set_header X-Forwarded-Proto https; # pass the Host: header from the client right along so redirects # can be set properly within the Rack application proxy_set_header Host $http_host; # we don't want nginx trying to do something clever with # redirects, we set the Host: header above already. proxy_redirect off; proxy_pass http://localhost:8080/demo-app-1.0/; } location /activiti-webapp-rest2/ { # an HTTP header important enough to have its own Wikipedia entry: # http://en.wikipedia.org/wiki/X-Forwarded-For proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # enable this if and only if you use HTTPS, this helps Rack # set the proper protocol for doing redirects: # proxy_set_header X-Forwarded-Proto https; # pass the Host: header from the client right along so redirects # can be set properly within the Rack application proxy_set_header Host $http_host; # we don't want nginx trying to do something clever with # redirects, we set the Host: header above already. proxy_redirect off; proxy_pass http://localhost:8080/activiti-webapp-rest2/; } location /activiti-webapp-explorer2/ { # an HTTP header important enough to have its own Wikipedia entry: # http://en.wikipedia.org/wiki/X-Forwarded-For proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # enable this if and only if you use HTTPS, this helps Rack # set the proper protocol for doing redirects: # proxy_set_header X-Forwarded-Proto https; # pass the Host: header from the client right along so redirects # can be set properly within the Rack application proxy_set_header Host $http_host; # we don't want nginx trying to do something clever with # redirects, we set the Host: header above already. proxy_redirect off; proxy_pass http://localhost:8080/activiti-webapp-explorer2/; } }

Locate the war artifacts at the server:

 - WorkflowEngineService - activiti-webapp-rest2 - activiti-webapp-explorer2 - demo-app

Copy the war to the the tomcat folder (/var/lib/tomcat7/webapps)
```
 sudo cp * /var/lib/tomcat7/webapps 
```

Test the installation. Open a Navigator and test the following URLs:

 http://134.60.64.132/demo-app-1.0/services/ http://134.60.64.132/activiti-webapp-explorer2/ http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments (basic authentication:kermit/kermit)

Test Cases

To execute the unit tests, we need to install the tool curl http://curl.haxx.se/ Optional: if you want to see the formatted response, the python library can be used. If not, you only need to remove this part: “| python -mjson.tool” of the command.

Create a new deployement:

It is needed to execute the following command:

 curl -X POST -F "file=@testCloudsocket.bpmn20.xml" -F "tenantId=ATOS" -F "endPointServiceList={\"workFlowId\":\"wf1\",\"endPointServices\":[{\"serviceId\":\"http://localhost:8080/demo-app-1.0/services/cardDesigner?wsdl\",\"serviceTaskId\":\"\",\"type\":\"WS\",\"ip\":\"127.0.0.1:8080\"},{\"serviceId\":\"http://localhost:8080/demo-app-1.0/services/email?wsdl\",\"serviceTaskId\":\"\",\"type\":\"WS\",\"ip\":\"127.0.0.1:8080\"}]}" --user kermit:kermit -H "content-type: multipart/form-data" http:// 134.60.64.132/activiti-webapp-rest2/service/repository/deployments | python -mjson.tool

Where the attributes are:

File is the bpm2.0 file (testCloudsocket.bpmn20.xml)
TenantId is the tenant of the organization (for the moment it is not used).

endPointServiceList is the list of the actual IP for the two services.

 Success response { "category": null, "deploymentTime": "2015-10-27T22:00:45.161Z", "id": "40", "name": "testCloudsocket.bpmn20.xml", "tenantId": "ATOS", "url": "http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40" }

Detail of the deployement

 curl --user kermit:kermit http:// 134.60.64.132/activiti-webapp-rest2/service/repository/deployments/{deploymentId} | python -mjson.tool in our case deploymentId is 40: curl --user kermit:kermit http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40 | python -mjson.tool

The description appears for your deployment.

 { "category": null, "deploymentTime": "2015-10-30T15:21:42.711+01:00", "id": "40", "name": "testCloudsocket.bpmn20.xml", "tenantId": "ATOS", "url": "http://127.0.0.1:8080/activiti-rest/service/repository/deployments/40" }

List of resources associated

 curl --user kermit:kermit http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/{deploymentId}/resources | python -mjson.tool in our case deploymentId is 40: curl --user kermit:kermit http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resources | python -mjson.tool

List of resources appears.

 [ { "contentUrl": "http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resourcedata/testCloudsocket.bpmn20.xml", "id": "testCloudsocket.bpmn20.xml", "mediaType": "text/xml", "type": "processDefinition", "url": "http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resources/testCloudsocket.bpmn20.xml" }, { "contentUrl": "http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resourcedata/testCloudsocket.testCloudsocket.png", "id": "testCloudsocket.testCloudsocket.png", "mediaType": "image/png", "type": "resource", "url": "http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resources/testCloudsocket.testCloudsocket.png" } ]

Get the associated resource

 curl --user kermit:kermit http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/{deploymentId}/resourcedata/testCloudsocket.bpmn20.xml

In our case deploymentId is the id 40:

 curl --user kermit:kermit http://134.60.64.132/activiti-webapp-rest2/service/repository/deployments/40/resourcedata/testCloudsocket.bpmn20.xml

The BPMN file associated to our deployment:

 <?xml version='1.0' encoding='UTF-8'?> <definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:activiti="http://activiti.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.activiti.org/processdef"> <import namespace="http://demo.seiplab.atosresearch.eu/" location="http://http://127.0.0.1:8080/demo-app-1.0/services/cardDesigner?wsdl" importType="http://schemas.xmlsoap.org/wsdl/"/> <import namespace="http://demo.seiplab.atosresearch.eu/" location="http://http://127.0.0.1:8080/demo-app-1.0/services/email?wsdl" importType="http://schemas.xmlsoap.org/wsdl/"/> <process id="testCloudsocket" name="testCloudsocket" isExecutable="true"> <userTask id="sid-9956FEED-377C-4C6D-81F5-1DF14EE36D56" name="test" activiti:candidateGroups="sales"/> <startEvent id="sid-AD365612-18DE-48F1-971F-944E51AB8804"/> <endEvent id="sid-F9B53F7F-5822-4653-BA91-CA88B3A026D3"/> <sequenceFlow id="sid-685E0C22-D2AD-4DF4-96C6-A435792C214A" sourceRef="sid-AD365612-18DE-48F1-971F-944E51AB8804" targetRef="sid-9956FEED-377C-4C6D-81F5-1DF14EE36D56"/> <sequenceFlow id="sid-541CE92A-8C33-4041-9359-693F38333D2F" sourceRef="sid-9956FEED-377C-4C6D-81F5-1DF14EE36D56" targetRef="sid-F9B53F7F-5822-4653-BA91-CA88B3A026D3"/> </process> <bpmndi:BPMNDiagram id="BPMNDiagram_testCloudsocket"> <bpmndi:BPMNPlane bpmnElement="testCloudsocket" id="BPMNPlane_testCloudsocket"> <bpmndi:BPMNShape bpmnElement="sid-9956FEED-377C-4C6D-81F5-1DF14EE36D56" id="BPMNShape_sid-9956FEED-377C-4C6D-81F5-1DF14EE36D56"> <omgdc:Bounds height="80.0" width="100.0" x="315.0" y="163.0"/> </bpmndi:BPMNShape> <bpmndi:BPMNShape bpmnElement="sid-AD365612-18DE-48F1-971F-944E51AB8804" id="BPMNShape_sid-AD365612-18DE-48F1-971F-944E51AB8804"> <omgdc:Bounds height="30.0" width="30.0" x="136.45001220703125" y="184.0"/> </bpmndi:BPMNShape> <bpmndi:BPMNShape bpmnElement="sid-F9B53F7F-5822-4653-BA91-CA88B3A026D3" id="BPMNShape_sid-F9B53F7F-5822-4653-BA91-CA88B3A026D3"> <omgdc:Bounds height="28.0" width="28.0" x="575.4500122070312" y="189.0"/> </bpmndi:BPMNShape> <bpmndi:BPMNEdge bpmnElement="sid-541CE92A-8C33-4041-9359-693F38333D2F" id="BPMNEdge_sid-541CE92A-8C33-4041-9359-693F38333D2F"> <omgdi:waypoint x="415.0" y="203.0"/> <omgdi:waypoint x="575.4500122070312" y="203.0"/> </bpmndi:BPMNEdge> <bpmndi:BPMNEdge bpmnElement="sid-685E0C22-D2AD-4DF4-96C6-A435792C214A" id="BPMNEdge_sid-685E0C22-D2AD-4DF4-96C6-A435792C214A"> <omgdi:waypoint x="166.44738152787627" y="199.2809153861509"/> <omgdi:waypoint x="315.0" y="202.06345112885748"/> </bpmndi:BPMNEdge> </bpmndi:BPMNPlane> </bpmndi:BPMNDiagram> </definitions>

Delete the deploment

 curl -X DELETE --user kermit:kermit http://127.0.0.1:8080/activiti-rest/service/repository/deployments/40

User Manual

API specification

HandBook

Handbook WFE

Developer manual

API specification 1.5

2017-03-08T14:51:03Z

API specification

We have extended the Activiti API, so you can find more information in the following link Activiti REST API

Deployment Phase

Create a new deployment (dinamic service tasks):
```
 POST repository/deployments 
```

The request body should contain data of type multipart/form-data. There should be exactly one file in the request, any additional files will be ignored. The deployment name is the name of the file-field passed in. If multiple resources need to be deployed in a single deployment, compress the resources in a zip and make sure the file-name ends with .bar or .zip.

Two additional parameters (form-field) can be passed in the request body:

tenantId: it is the id of the tenant that will be associated to the deploy business process.
endPointServiceList: It is the list of the actual URL (IP/host_name and the port) of the associated services. The parameter has to have the following structure:

EndPointServiceBundle: contain all the necessary information to modify the bpmn with the actual service IPs

workFlowId(literal): Identify the workflowId (set as "id" attribute on process element)
workFlowName(literal): Set a workflow name (set as "name" attribute in process element)
endPointServices (EndPointServices[]): list of the actual IPs of services, it is described by the EndPointServices entity.

EndPointServices: Include the details to identify the services and contains the real IP.

serviceId (literal): identification of the services.
type (literal): Only accepted two values: WS (Web services) and REST.
serviceTaskId (literal): Indicate the identifier of the service task, associated to this serviceId. It is optional since there are some situation where the description of the services are not associated directly to the service task.
Ip (literal): It indicates the IP/Name of the host plus the port, following the pattern <ip/host_name>:<port>. If the port is 80 o 443 it is not necessary to include them in this attribute.

Example of the json file:

 { "workFlowId":"wf1", "workFlowName": "a name", "endPointServices":[ { "serviceId":"http://localhost:8080/demo-app-1.0/services/cardDesigner?wsdl", "serviceTaskId":"", "type":"WS", "ip":"127.0.0.1:8080" }, { "serviceId":"http://localhost:8080/demo-app-1.0/services/email?wsdl", "serviceTaskId":"", "type":"WS", "ip":"127.0.0.1:8080" } ] }

Note: Change the ip attribute with the available Web Service definition (running WSDL description), if not the WorkflowEngine doesn’t allow to deploy the BPMN file. Response codes:

 *201 Indicates the deployment was created. *400 Indicates there was no content present in the request body or the content mime-type is not supported for deployment. The status-description contains additional information.

Success response body:

 { "category": null, "deploymentTime": "2015-10-30T15:21:42.711+01:00", "id": "40", "name": "withEmail.bpmn20.xml", "tenantId": "ATOS", "url": "http://127.0.0.1:8080/activiti-rest/service/repository/deployments/40" }

Execution Phase

List of instances

 GET service/runtime/process-instances?deploymentId=<deployemtId>

This action obtains all the process instances associated to one deployment.

The following parameters need to be indicated:

deployemtId: the identifier of the deployment in order to obtain its instances.

Response codes:

 *200 Ok. *400 Indicates there was no content present in the request body or the content mime-type is not supported for deployment. The status-description contains additional information.

The response message contains the list of instances with the UrL parameters, in this case all the instances for the specific deployment (deployemtId)

Example of the response, list of instances:

 { "data": [ { "activityId": "sid-2B0D606E-2AF8-4EF6-AFF0-E137471C6990", "businessKey": null, "completed": false, "deploymentId": "152501", "ended": false, "id": "152505", "processDefinitionId": "CloudSocketSendingInvoicesFinal_withTokenActiviti:1:152504", "processDefinitionUrl": "http://localhost:8080/activiti-webapp-rest2/service/repository/process-definitions/CloudSocketSendingInvoicesFinal_withTokenActiviti:1:152504", "suspended": false, "tenantId": "", "url": "http://localhost:8080/activiti-webapp-rest2/service/runtime/process-instances/152505", "variables": [] }, { "activityId": "sid-2B0D606E-2AF8-4EF6-AFF0-E137471C6990", "businessKey": null, "completed": false, "deploymentId": "152501", "ended": false, "id": "152515", "processDefinitionId": "CloudSocketSendingInvoicesFinal_withTokenActiviti:1:152504", "processDefinitionUrl": "http://localhost:8080/activiti-webapp-rest2/service/repository/process-definitions/CloudSocketSendingInvoicesFinal_withTokenActiviti:1:152504", "suspended": false, "tenantId": "", "url": "http://localhost:8080/activiti-webapp-rest2/service/runtime/process-instances/152515", "variables": [] } ], "order": "asc", "size": 2, "sort": "id", "start": 0, "total": 2 }

Adaptation for one process instance

 POST service/runtime/process-instances/<instanceId>/adaptation

This action allows to modify de endpoints of the service tasks for the instance indicated, for example, if the adaptation engine realizes that something happens through the Cloud Provider Engine could call the WF engine to modify the conditions The following actions are included in order to configure the new environment:

Suspend the instance.
Base on this instance recover the associated deployment and obtain the original BPMN file.
Base on this original BPMN, the system parses the new endpoints (detailed in a json message), besides, we can change the id and name of the new workflow definition (introducing the appropriated attributes in the json ).
Create a new deployment with the new BPMN applying the new conditions and change the name of the process definition accordingly with the id of the new workflow.
Obtain the new process description (associating to his new deployment)
Update the instance with the new process description.
Active the instance again.

The following parameters need to be indicated:

InstanceId: this id of the instance. It is indicated in the URL when the call is created.
endPointServiceList: It is the list of the new URLs (IP/host_name and the port) of the associated services. The parameter has to have the following structure:

EndPointServiceBundle: contain all the necessary information to modify the bpmn with the actual service IPs

workFlowId(literal): Identify the new workflowId after the adaptation. If it is not indicated, the system maintain the previous id + “_adaptation”
workFlowName(literal): Set the new workflow name (set as "name" attribute in process element) after the adaptation. If it is not indicated, the system maintain the previous name + “_adaptation”
endPointServices (EndPointServices[]): list of the actual IPs of services, it is described by the EndPointServices entity.

EndPointServices: Include the details to identify the services and contains the real IP.

serviceId (literal): identification of the services.
type (literal): Only accepted two values: WS (Web services) and REST.
serviceTaskId (literal): Indicate the identifier of the service task, associated to this serviceId. It is optional since there are some situation where the description of the services are not associated directly to the service task.
Ip (literal): It indicates the IP/Name of the host plus the port, following the pattern <ip/host_name>:<port>. If the port is 80 o 443 it is not necessary to include them in this attribute.

Example of the json file:

 {"workFlowId":"WP3_test_Final_AfterAdaptation", "workFlowName":" WP3_test_Final_AfterAdaptation_Name", "endPointServices":[ { "serviceId":"http://localhost:8080/demo-app-1.0/services/cardDesigner?wsdl", "serviceTaskId":"", "type":"WS", "ip":"127.0.0.1:8080" }, { "serviceId":"http://localhost:8080/demo-app-1.0/services/email?wsdl", "serviceTaskId":"", "type":"WS", "ip":"127.0.0.1:8080" } ] }

Note:

Change the ip attribute with the available Web Service definition (running WSDL description), if not the WorkflowEngine doesn’t allow to deploy the BPMN file.
The attributes workFlowId and workFlowName can be empty, in this case the name will be the same.
TenantId is not necessary since, we will use the same tenantId that the original deployment has.

Response codes:

 *201 Indicates the adaptation for this instance was created. *400 Indicates there was no content present in the request body or the content mime-type is not supported for deployment. The status-description contains additional information.

Success response body:

"activityId": it indicates the task where the instance is stopped (we are validating that the new process definition contains this sid),
"businessKey": (To be validated),
"completed": it indicates if the instance has completed,
"ended": it indicates if the instance has ended,
"id": id of the instance, of course it is the same that we have at the beginning (InstanceId in the URL call),
"processDefinitionId": New process definition with the new endpoints, id and name,
"processDefinitionUrl": url to obtain the process definition,
"suspended": if the instance is suspended,
"tenantId": the tenant id, we will maintain the original one,
"url": url to obtain the instance,
"variables": the variables associated to this instance.

Example of the response:

 { "activityId": "sid-FD0DD857-8763-4E46-8ED4-530FFAF19B27", "businessKey": null, "completed": false, "ended": false, "id": "82501", "processDefinitionId": "WP3_test_Final_AfterAdaptation:1:85004", "processDefinitionUrl": "http://localhost:8080/activiti-webapp-rest2/service/repository/process-definitions/WP3_test_Final_AfterAdaptation:1:85004", "suspended": false, "tenantId": "", "url": "http://localhost:8080/activiti-webapp-rest2/service/runtime/process-instances/82501", "variables": [] }

Handbook WFE 1.0

2017-03-08T12:26:05Z

TO be completed

Synergic Cross-Layer Monitoring Framework 1.1

2017-02-14T15:31:40Z

Research Problem #

Various monitoring frameworks have been proposed and offered which usually cover only one or two
levels of abstraction. Moreover, such frameworks usually rely on level-independent mechanisms
which are able to support the individual assessment over the respective metrics. The latter
leads to the issue of measurability gaps: as adjacent levels are not able to cooperate,
then it is not possible to produce measurements of a high-level metric from measurements
of a lower-level metric. This issue leads to the fact that most of the monitoring frameworks
are able to support a rather limited and fixed set of metrics. Related to this issue is
the problem of metric computability: in particular, the frameworks are not able, based on the
high-level but complete description of a metric, to produce its measurements. They rather
stay on a quite technical mapping of the fixed description of the limited set of metrics
supported to the corresponding sensing or aggregation code needed for the metric measurement.

The above inabilities of the current monitoring frameworks highlight the need for introducing
a cross-layer monitoring framework which is able to measure metrics over different levels as
well as cover measurability gaps by being able to address the measurement of any kind of metric.
Such a framework should also rely on a flexible and complete high-level description of a metric
which is independent from any technical specificities.

Solution #

FORTH and University of ULM (UULM) have developed two separate monitoring frameworks which cover different
levels of abstraction. The monitoring framework of FORTH covers mainly the workflow and service level,
while the monitoring of UULM covers mainly the infrastructure and platform levels. As such, it
is apparent that the integration of these frameworks can result in the production of a synergic
framework which is able to cover the measurement of metrics at all possible levels. To this end,
a synergic framework has been designed and is currently developed which is able to capitalize over
the individual capabilities of the aforementioned monitoring frameworks. Each individual framework
is seen as a service which offers monitoring and evaluation capabilities. The cross-layer aggregation
of metric measurements between the two individual frameworks is facilitated by a publish-subscribe
mechanism: this means that one framework subscribes to those lower-level measurements that are needed in order to
aggregate a measurement for a higher-level. The combined framework also offers another publish-subscribe
mechanism in order to enable interested components, like an Adaptation Engine, to retrieve the SLO
evaluations produced and enact, in this case, respective adaptation actions.

Other interesting features of the combined framework which map to common features of the individual
frameworks are the following: recovery capability in case a monitoring component fails, data
redundancy to avoid important monitoring data loss as well as capability to monitor both domain-independent
and domain-specific metrics.

Architecture #

The architecture of the combined/synergic monitoring framework can be seen in the following figure. As
expected, the respective individual frameworks are included and they communicate via a publish-subscribe
mechanism. In addition, this mechanism is used in order to propagate measurements to the Evaluator
component which includes a CEP engine responsible for evaluating conditions over the propagated measurements.
The events produced are further propagated to interested components, such as Adaptation Engines.
The measurements can also be inspected in a programmatic or UI-based manner by two other components
which are part of the Collosseum cloud orchestration engine. The Collosseum API enables the
programmatic retrieval of measurements that are stored in the Evaluator while the Collosseum UI enables the
production of graphs over these measurements. Both components can rely on the measurements propagated
by the individual monitoring frameworks but in principle they could also require the further aggregation
of the stored measurements which is natively supported by the encompassing CEP Engine.

This innovation item will be made available at Gitlab.

References: #

Synergic Cross-Layer Adaptation Framework 1.1

2017-02-14T14:53:16Z

Research Problem #

BPaaS adaptation needs to be performed across different levels of abstraction. This is required
in order to avoid having different adaptation mechanisms being triggered
independently such that the effects of one mechanism can be undone by the other, thus resulting in
vicious adaptation cycles. Moreover, different levels need to be covered as one problematic
situation could be impossible to be addressed by just one level. For instance, the migration of
a software component from one cloud to another might also require adopting the workflow level in
order to perform a respective replacement of the service endpoint in the workflow description,
usually done in the context of service replacement.

Solution #

The cross-layer BPaaS adaptation solution comprises the integration of different adaptation
frameworks that have been developed by two academic partners of the CloudSocket project, namely
FORTH and University of ULM. By joining forces, current adaptation implementation gaps at
different levels are covered in the form of services which include methods mapping to the level-specific
adaptation actions. In addition, the respective individual features offered by
each framework can be capitalised and exploited to make them available for the combined framework.
As such, this solution looks ideal for realising or replacing the functionality of the
Adaptation Engine in the current CloudSocket implementation.

Architecture #

The architecture of the combined / synergic framework is depicted in the following figure.
As it can be seen, this framework comprises the following components: (a) the Transformer is responsible for mapping adaptation rules, which are
generated by the BPaaS Evaluation Environment, into two forms: one which is
suitable for storage in the Rule Base and another which is suitable for storage in the Adaptation
Engine; (b) the Rule Base is responsible for storing the adaptation rules provided by the
Transformer and discovering the one which is the most suitable to confront the current
problematic situation. The latter discovery is supported via the use of monitoring facts
communicated by the (BPaaS) Monitoring Framework. In case that an adaptation rule should be
triggered, then the Adaptation Engine is informed in order to enact the respective adaptation;
(c) the Adaptation UI is a UI enabling the expert to modify the rules automatically generated
by the BPaaS Evaluation Environment or to insert new rules to cover new adaptation situations;
(d) the Adaptation Engine is a normal workflow engine which is able to enact the adaptation
workflows needed while it is also responsible for storing the adaptation actions that have been
performed in the Adaptation DB; (e) the workflows executed by the Adaptation Engine map to
method calls of level-specific services: the service and workflow level is covered by the
cross-layer adaptation framework of FORTH while the infrastructure and platform level is
covered by the adaptation framework of UULM. As already indicated, the methods of these
services map to particular level-specific adaptation actions that are offered for
execution; (f) finally, the Adaptation Service enables the retrieval of adaptation history
information from the Adaptation DB, mostly for the analysis purposes of the BPaaS Evaluation
Environment, while it also enables the programmatic execution of adaptation workflows, in case
where manual intervention is needed, thus mapping to the case that the current situation is
not covered by the current set of adaptation rules modelled.

The source code will be available at the Gitlab.

References: #

PaaS Orchestration and Adaptation 1.7

2017-02-07T12:22:03Z

1 Problem Description
2 Solution Approach
3 Architecture
4 Usage
5 Video Presentation
6 References

Problem Description

PaaS abstraction is still an open and on-going topic in current research. It can drastically decrease costs, by e.g. sharing infrastructure of public PaaS providers to host an application. Therefore, the integration of a PaaS abstraction into the Cloud Provider Engine is of major interest for CloudSocket. See Deliverable D3.3 for a research on current Cloud Orchestration Tools and their state towards PaaS services and provider integration.

Solution Approach

Our solution for this is two-fold: (i) enhancing the application and component instance life-cycle by PaaS-specific activities, and (ii) integrating and developing the PaaS Unified Library (PUL) into the Cloud Provider Engine, which is powered by Colosseum.

Architecture

The communication to the cloud provider is ensured by the PaaS abstraction layer “Dagger”, which is the higher-level pendant to Sword, the IaaS abstraction layer. This layer unifies the API calls to different Cloud providers.

Usage

http://134.60.64.166/cloud-provider-engine-ui/#/login

Video Presentation

A video is available on YouTube.

References

For more details on this, please check the respective repositories:

Research branch of Cloudiator/Colosseum

Adaptation Management component repository

Also please refer to the wiki pages:

Adaptation Management

Unified PaaS component

Adaptation Management 1.4

2017-02-07T11:35:06Z

1 Description
2 Architecture
3 References
4 Link

Description

This component manages and processes adaptations plans. Those are the executable equivalent to the CAMEL definitions of Adaptation Rules and Adaptation Tasks.

Architecture

The current Adaptation Management component consists of a REST-interface connected to a server that stores the adaptation plans. A plan is a represented as connected steps of a workflow.

[Figure REST interface / example]

[Figure communication AM, AM-C, AM-Test]

References

Deliverable D3.3 Deliverable D3.4

Link

https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/adaptation-management https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/adaptation-management-test https://omi-gitlab.e-technik.uni-ulm.de/cloudsocket/adaptation-management-client

AXE Adaptation Framework 1.0

2017-02-07T08:37:28Z

1 Research Problem
2 Solution Approach
3 Architecture
4 References
5 Link

Research Problem

Current Cloud Orchestration Tools have either too simplistic or too verbose methods of adaptation descriptions. Commercial tools have mainly simple approaches of adaptation, such as container scaling, and research-driven tools suffer of too loosely described methods that are hardly usable for the average customer. An example for the latter is the BPMN integration into TOSCA to describe workflow, for which the user has to implement the classes on their own.

Solution Approach

We aim for an integrated component, that holds the CAMEL adaptation plans, which targets our requirements as described in D3.3. We are sure that we achieve a more useable approach than generic BPMN workflows, as our approach is very specific to the cloud. The Axe adaptation approach furthermore targets to be integrated as loosely coupled component that can be added to the Cloud Provider Engine / Cloudiator without changing internals.

Architecture

[Picture of Adaptation Management, Axe [MS] and how they communicate] The Adaptation Management component was integrated with the Axe component of the Cloudiator framework in a loosely coupled way. The figure above shows the integration into the components. As any other external component it uses the REST interface of Colosseum and changes the entities as described in the adaptation plans.

References

Deliverable D3.3 Deliverable D3.4

Link

Adaptation Management

OWL-Q Parser 2.0

2017-02-03T18:36:35Z

1 Research Problem
2 Solution Approach
3 References

Research Problem

Due to the heterogeneity, both structural and semantic, in QoS term description which is inherent in the service and cloud world, there is a need for introducing a semantic language which is able to cover the non-functional service description. As such, the use of such language will enable reaching better accuracy levels in service discovery and composition. Apart from non-functional service profiles and requests, this language should be able to specify SLAs in the way that extends the current state-of-the-art. This means that it should be complete enough in order to cover all information required for supporting the management of all service lifecycle activities.

The required language should be able to also specify KPIs as well as link them to goals in order to support BPaaS evaluation as well as goal analysis enabling broker to inspect whether their strategic, tactical and operational goals are satisfied. The current set of languages employed for this purposes suffer from the following drawbacks: (a) they are not rich or complete enough to cover the respective domain; (b) they are mapped to a quite technical level which is not suitable for business experts; (c) they are not able to exploit different information sources, even external to the respective BPaaS (management) system; (d) in some cases, they are not linked to goals.

Solution Approach

OWL-Q is a language which has been considered one of the most prominent non-functional service specification languages by survey papers in service quality description. This is due to its expressivity, richness and completeness (in terms of domain coverage) as well as to the fact that it relies on semantics and ontologies in particular. OWL-Q extends the W3C OWL standard and has been designed into several facets, each one focusing on different measurability aspects, including attribute, metric, unit (of measurement), and value type. It is also assorted by: (a) semantic rules specified in the SWRL language (W3C Submission) which enable the validation of OWL-Q models as well as the derivation of new knowledge that can take even the form of matches between quality terms. An overview of OWL-Q can be seen in the following two figures, where the first figure covers the 5 facets and the second the last one.

OWL-Q has been recently extended in order to also specify SLAs as well as cover the previously referred research gap in SLA description. In a nutshell, the OWL-Q SLA extension, named as Q-SLA, has the following capabilities: (a) it can specify both SLA and SLA templates; (b) it captures directly the notion of a Service Level (SL) to be delivered by a service which maps to a logical combination of Service Level Objectives (SLOs). The latter are simple constraints over quality terms; (c) it enables the transitioning from one SL to another in order to allow for flexibility in SLA management. In particular, such a SL transitioning can be beneficial for a signatory party as it enables the on-demand modification of the SL to be delivered without the need of performing a critical action, such as SLA re-negotiation. Such a modification might be required when specific party requirements are modified, e.g., due to increase in workload due to an increase of the party clientelle; (d) it associates an SLO to various important concepts, like qualifying conditions, dictating when the SLO should hold, as well as penalties and rewards to be supplied when the respective SLO is violated or surpassed, respectively; (e) it is able to specify service price models and connect then to the SLO penalties and rewards; (f) it enables the hierarchical specification of SLAs which is highly needed in the case of BPaaS services, which are cross-level agglomeration of cloud services; (g) finally it allows the definition of critical situations for settlement along with the respective actions that need to take place, such as SLA re-negotiation or cancelling. As Q-SLA builds on top of the OWL-Q specification facet, an overview of this extension is depicted in the following figure which uses different colours to characterise concepts which are original or new with respect to the OWL-Q language.

By again focusing on the respective research gap identified previously, another OWL-Q extension was developed in order to cover the modelling of KPIs and goals as well as their linkage. This extension is depicted in the following figure where different colours (red vs. other) are being used to denote extension or original OWL-Q concepts. In summary, the following features characterise this OWL-Q extension: (a) it specifies KPIs as extensions to simple constraints which are enhanced with the description of additional thresholds (warning apart from violation ones); (b) it enables issuing queries over databases or calling APIs and using the respective result as an input argument to a KPI metric formula; (c) it enables specifying direct relationships between KPIs (child-parent) in order to enable KPI drill-down to discover the root causes of a high-level KPI violation; (d) it covers the modelling of manual measurements as humans are usually another source of information apart from the automatic ones; (e) it allows the modelling of KPI evaluations which can assist in the checking of respective trends in BPaaS performance; (f) it covers the specification of goal hierarchies as well as the linking of goals to KPIs via goal contributions.

Both OWL-Q as well as its extensions are supported by a JAVA programmatic API which enables the parsing as well as the serialisation of OWL-Q models. This API can be used to construct non-functional service matchmaking and selection algorithms as well as customised OWL-Q editors. For the time being, the latter type of editor does not yet exist but is planned to be developed. In case the interested reader and possible adopter of OWL-Q desires to manually edit an OWL-Q model, then he/she can use well-known ontology editors, like Protege, for now.

The source code is publicly available at GitLab.