TOSCA
Introduction
The Topology and Orchestration Specification for Cloud Applications (TOSCA) is an OASIS open standard that provides language to describe a topology of cloud based web services, their components, relationships, and the processes that manage them. TOSCA provides mechanisms for abstraction and composition, thereby enabling portability and automated management across cloud providers regardless of underlying platform or infrastructure.
The TOSCA specification allows the user to define a Service Template which describes an online application or service and how to deploy it.
Core TOSCA Concepts
The figure below illustrates the core components of a TOSCA service template. At the heart of a TOSCA service template is a Topology Template that describes a model of the service’s online resources, including how they connect together and how they should be deployed. These resources are represented by a Node Template. The Node Template and a Relationship Template are the building blocks of any TOSCA Specification.
Node templates describe the resources that will be instantiated when the service template is deployed. They are linked to other nodes through relationships.
Relationship templates can be used to provide additional information about those relationships, for example how to establish a connection between two nodes.
Interfaces are a collections of user-defined operations that are invoked by a TOSCA orchestrator like Unfurl. TOSCA defines a standard interface for lifecycle management operations (creating, starting, stopping and destroying resources) and the user can define additional interfaces for “Day Two” operations, such as maintenance tasks.
Artifacts such as container images, software packages, or files that need to be deployed or used as an implementation for an operation.
Workflows allows you to define a set of manually defined operations to run in a sequential order.
Type definitions TOSCA provides an object-oriented type system that lets you declare types for all of the above components as well as custom data types that provide validation for properties and parameters.
See also
For more information, see the full TOSCA Language Reference and the Glossary section.
Service Template
A TOSCA service template contains all the information needed to deploy the service it describes. In Unfurl, a service template can be a stand-alone YAML file that is included in the ensemble.yaml configuration file or embedded directly in that file as a child of the Service templates element.
A service template has the following sections:
Metadata sections, which includes the
tosca_definitions_version,description,metadata,dsl_definitionsImports and Repositories sections
Types sections that contain types of Node, Relationships, Capabilities, Artifacts, Interfaces, Policy and Groups
Topology Template which include sections for Inputs, Outputs, Node and relationship templates, Substitution Mappings, Groups, policies and Workflows.
Example
tosca_definitions_version: tosca_simple_unfurl_1_0_0 # or use the standard tosca_simple_yaml_1_3
description: An illustrative TOSCA service template
metadata: # the following metadata keys are defined in the TOSCA specification:
template_name: hello world
template_author: onecommons
template_version: 1.0.0
repositories:
tosca-community-contributions:
url: https://github.com/oasis-open/tosca-community-contributions.git
imports:
- file: my-shared-types.yaml
namespace_prefix: base # optional
- file: profiles/orchestration/1.0/profile.yaml
repository: tosca-community-contributions
node_types:
# ... see the "types” section below
topology_template:
# ... see the “topology_templates” section below
node_templates:
# ... see the node_templates section below
relationship_templates:
# ... see the relationship_templates section below
Types
Every entity in TOSCA (including Nodes, Relationships, Artifacts and Data) has a declared type. custom type hierarchies can be defined in the Service Template.
Types declare the required properties, default definitions, and interface operations for an entity. Each type of entity has can have its own section in the service template, for example, node_types, relationship_types, data_types, artifact_types, interface_types, etc. but type names all are share one namespace in the YAML document or Python module they are defined in.
Example
This example defines a node type named “MyApplication” that inherits from the “tosca.nodes.SoftwareComponent” node type.
node_types:
MyApplication:
derived_from: tosca.nodes.SoftwareComponent
attributes:
private_address:
type: string
properties:
domain:
type: string
default: { get_input: domain }
ports:
type: tosca.datatypes.network.PortSpec
requirements:
- host:
capabilities: tosca.capabilities.Compute
relationship: tosca.relationships.HostedOn
- db:
capabilities: capabilities.postgresdb
relationship: tosca.relationships.ConnectsTo
interfaces:
# TOSCA defines Standard interface for lifecycle management but you can define your own too
Standard:
create: create.sh
configure: configure.sh
delete: delete.sh
Topology Template
The topology Template defines the components of the service being deployed. It can be thought of as a graph of node templates and other components along with their relationships and dependencies.
Topologies can parameterized with Inputs, define outputs, and contains Node Templates, Relationship Templates, Groups, policies, and Workflows.
Topologies can be embedded in other topologies via Substitution Mappings.
Example
topology_template:
inputs:
domain:
type: string
default: example.com
outputs:
url:
type: string
# value: { concat: [ https://, { get_input: domain }, ':', { get_attribute: [ myapp, portspec, source ] }, '/api/events'] }
# # Unfurl also support ansible-enhanced jinja2 template so you could write this instead:
value: https://{{ TOPOLOGY.inputs.domain }}:{{ NODES.myApp.portspec.source }}/api/events
from tosca import Eval, TopologyInputs, TopologyOutputs
class Inputs(TopologyInputs):
domain: str = "example.com"
class Outputs(TopologyOutputs):
url: str = Eval(
"https://{{ TOPOLOGY.inputs.domain }}:{{ NODES.myApp.portspec.source }}/api/events"
)
Deployment blueprints
A deployment blueprint is an Unfurl extension that allows you to define blueprint that is only applied when its criteria matches the deployment environment. It inherits from the main topology template and node templates with matching names replace the ones in the topology template’s. For example, see this example.
Node Template
A Node Template defines a node that gets instantiated into an instance (or resource) when the service template is deployed. A node template uses its node type as a prototype and can define properties, capabilities, artifacts, and requirements specific to the template.
Example
node_templates:
myApp:
type: myApplication
artifacts:
image:
type: tosca.artifacts.Deployment.Image.Container.Docker
file: myapp:latest
repository: docker_hub
requirements:
- host: compute
- db:
node: mydb
relationship: mydb_connection
mydb:
type: PostgresDB
properties:
name: mydb
compute:
type: unfurl.nodes.Compute
capabilities:
host:
properties:
num_cpus: 1
disk_size: 200GB
mem_size: 512MB
Requirements and Relationships
Requirements let you define relationships between nodes and must be assigned a relationship type. Requirements are declared as a list on the node template, allowing multiple requirements with the same name, whose number is constrained by the occurrences keyword in the requirement’s definition.
Relationship types can have properties and have interfaces and operations associated with them. For example, the orchestrator will use operations defined for the built-in Configure interface to create or remove the relationship.
Requirements declared on a node type define the constraints for that requirement, for example the type of node it can target or the number of target matches each node template can have for that requirement.
Requirements declared on a node template define the node that the requirement is targeting, either directly by naming that node template, or by defining the search criteria for finding the match, for example by including a node_filter.
So when the node field refers to an node type in requirement defined on a node template its defining a search criteria for finding a matching node template, but when the requirement is on node type, it is only defining a validation constraint where the node template’s match must implement that type – it won’t use that field to search for a match.
Relationship templates can be declared to provide specific information about the requirement’s relationship with its target node. Relationship templates can be defined directly inline in the requirement or they can be declared separately in the relationship_templates section of the topology and referenced by name.
Examples
This simple example using built-in TOSCA types that defines the “host” requirement as using the tosca.relationships.hostedOn relationship, so we can define the requirement with just the name of the target node template:
topology_template:
node_templates:
myApp:
type: tosca.nodes.SoftwareComponent
requirements:
- host: another_node
The following more complex example first defines a “MyApplication” node type that requires one “db” relationship of type “DatabaseConnection”. Then it defines a node template (“myApp”) with that node type with a “db” requirement that uses a node filter to find the matching target node and a named relationship template of type “DatabaseConnection” with properties for connecting to the database.
node_types:
MyApplication:
derived_from: tosca.nodes.SoftwareComponent
requirements:
- db:
node: tosca.nodes.DBMS
relationship: DatabaseConnection
occurrences: [1, 1]
relationship_types:
DatabaseConnection:
derived_from: tosca.relationships.ConnectsTo
properties:
username:
type: string
password:
type: string
metadata:
sensitive: true
topology_template:
relationship_templates:
mydb_connection:
type: DatabaseConnection
properties:
username: myapp
password:
eval:
secret:
myapp_db_pw
node_templates:
myApp:
type: MyApplication
requirements:
- db:
relationship: mydb_connection
node_filter:
properties:
- name: mydb
import tosca
from tosca import Eval, Node, Property, Relationship
class DatabaseConnection(tosca.relationships.ConnectsTo):
username: str
password: str = Property(metadata={"sensitive": True})
class MyApplication(tosca.nodes.SoftwareComponent):
db: "DatabaseConnection | tosca.nodes.DBMS"
mydb_connection = DatabaseConnection(
"mydb_connection",
username="myapp",
password=Eval({"eval": {"secret": "myapp_db_pw"}}),
)
myApp = MyApplication(
"myApp",
db=mydb_connection,
)
Artifacts
An artifact is an entity that represents an asset such as files, container images, or software packages that need to be deployed or used by an implementation of an operation. Like other TOSCA entities, artifacts have a type and can have properties, attributes, and operations associated with it.
Artifacts are declared in the artifacts section of node templates and node types, and can refer to the repository that it is part of.
This example defines an artifact that is a container image, along with a repository that represents the image registry that manages it:
docker_hub:
url: https://registry.hub.docker.com/
credential:
user: user1
token:
eval:
secret:
dockerhub_user1_pw
topology_template:
node_templates:
myApp:
type: MyApplication
artifacts:
image:
type: tosca.artifacts.Deployment.Image.Container.Docker
file: myapp:latest
repository: docker_hub
Artifacts can be used in the following ways:
An operation’s implementation's,
primaryfield can be assigned an artifact which will be used to execute the operation. Artifacts derived fromunfurl.artifacts.HasConfiguratorwill use configurator set on its type, otherwise it will treated as a shell script unless theclassNamefield is set in the implementation.An implementation can also list artifacts in the
dependenciesfield which will be installed if necessary.The get_artifact TOSCA function to reference to artifact’s URL or local location (if available).
An artifact and its properties can be accessed in Eval Expressions via the .artifacts key. (see Artifact enhancements) or as
Nodeattributes when using the Python DSL.
Artifacts that are referenced in an operation’s implementation will be installed on the operation’s operation_host (by default, where Unfurl is running) as part of the Job Lifecycle if the artifact has Standard operations (create or configure) defined for it.
See also
For more information, refer to TOSCA Artifact Section
Interfaces and Operations
An Operations defines an invocation on an artifact or configurator. A TOSCA orchestrator like Unfurl instantiates a service template by executing operations. An operation has an Implementation, inputs, and outputs.
Conceptually an operation is comparable to method in an object-oriented language where the implementation is the object upon which the method is invoked, the inputs are the method’s arguments, and its outputs are the method’s return values.
An Interface is a collections of operations that can be defined in the interfaces section on TOSCA types or directly on TOSCA templates for nodes, relationships, and artifacts.
Example
interfaces:
Standard: # this is a built-in interface type so the "type" field is not required
inputs:
# inputs defined at this level will be made available to all operations
foo: bar
operations:
# in the simplest case an operation can just be the name of an artifact or shell command
create: ./myscript.sh
# a more complex operation:
configure:
implementation:
primary: my_ansible_playbook
dependencies:
# unfurl will install artifacts listed as dependencies if they are missing
- a_ansible_collection
# Unfurl extensions:
className: Ansible # the configurator class to use (redundant in this example)
environment: # environment variables to set when executing the operation
AN_ENV_VAR: 1
inputs:
foo: baz # overrides the above definition of "foo"
outputs:
an_ansible_fact: an_attribute
# an Unfurl extension for specifying the connections that the operations need to function:
requirements:
- unfurl.relationships.ConnectsTo.GoogleCloudProject
Operations can be invoked in the following ways:
The built-in Standard and Configure interfaces are invoked during deploy and undeploy workflows
Unfurl’s built-in Install are invoked for check and discovery.
Custom operations can be invoked with run workflow.
Operations can be invoked directly by custom Workflows, the Delegate configurator, or using Unfurl’s Python apis.
Implementation
As the example above illustrates, an operation’s implementation field describes which artifacts or configurator (Unfurl’s plugin system for implementing operations) to use to execute the operation.
As an Unfurl extension, an implementation section can also include an environment key with Environment Variables directives and a className field to explicitly name the configurator.
Inputs
Inputs are passed to the implementation of the operation. A TOSCA type can (optionally) define the expected inputs on a operation in much like a property definition. In Unfurl, inputs are evaluated lazily as needed by the operation’s configurator. Default inputs can defined directly on the interface and will be made available to all operations in that interface. If inputs are defined on multiple types in a type hierarchy for the same operation, they are merged together along with any inputs for that operation defined on directly on the template.
Outputs
An operation can define an attribute mapping that specifies how to apply the operation’s outputs. The meaning of keys in the mapping depends on the operation’s configurator, for example, a Ansible fact or a Terraform output.
If mapping’s value is a string, it names the attribute on the instance where the output will be saved.
If the value is null, no attribute mapping will be made but the output will be available to the resultTemplate and saved in the ensemble.
Interface types
Interface types define names of the operations in an interface along with their inputs and outputs. TOSCA defines a built-in interface types for lifecycle management operations and additional interface types can be declared in the service template for “Day Two” operations, such as maintenance tasks.
For example, Unfurl adds a built-in interface type for discovering resources, which it defines as:
interface_types:
unfurl.interfaces.Install:
derived_from: tosca.interfaces.Root
operations:
check:
description: Checks and sets the status and attributes of the instance
discover:
description: Discovers current state of the current instance and (possibly) related instances, updates the spec as needed.
revert:
description: Restore the instance to the state it was original found in.
connect:
description: Connect to a pre-existing resource.
restart:
description: Restart the resource.
Built-in interfaces
TOSCA defines two built-in interface types that are invoked to deploy a topology: the Standard interface for node templates and Configure interface for relationships templates.
The former defines lifecycle management operations (creating, starting, stopping and destroying resources) that are invoked when creating and deleting node instances and latter for configuring the Requirements between nodes.
It is not a requirement to define every operation for every template; the operation will be silently skipped if not defined. With the Standard interface, only the configure operation needs to be defined to instantiate a node, if the create operation isn’t defined, Unfurl will assume that configure will both create and configure the resource (and reconfigure when updating an existing deployment).
In addition, Unfurl provides a built-in Install interface which is invoked when running check and discover workflows.
Complete Example
Combining the above examples into one file, we have a complete service template:
tosca_definitions_version: tosca_simple_unfurl_1_0_0 # or use the standard tosca_simple_yaml_1_3
description: An illustrative TOSCA service template
metadata: # the following metadata keys are defined in the TOSCA specification:
template_name: hello world
template_author: onecommons
template_version: 1.0.0
repositories:
docker_hub:
url: https://registry.hub.docker.com/
credential:
user: user1
token:
eval: # eval is an Unfurl extension
secret: dockerhub_user1_pw
relationship_types:
DatabaseConnection:
derived_from: tosca.relationships.ConnectsTo
properties:
username:
type: string
password:
type: string
metadata:
sensitive: true
node_types:
MyApplication:
derived_from: tosca.nodes.SoftwareComponent
attributes:
private_address:
type: string
properties:
domain:
type: string
default: { get_input: domain }
ports:
type: tosca.datatypes.network.PortSpec
requirements:
- host:
capability: tosca.capabilities.Compute
relationship: tosca.relationships.HostedOn
- db:
relationship: DatabaseConnection
interfaces:
# TOSCA defines Standard interface for lifecycle management but you can define your own too
Standard:
create: create.sh
configure: configure.sh
delete: delete.sh
topology_template:
inputs:
domain:
type: string
outputs:
url:
type: string
value:
{
concat:
[
https://,
{ get_input: domain },
/api/events
],
}
# Unfurl also support ansible-enhanced jinja2 template so you could write this instead:
# value: https://{{ TOPOLOGY.inputs.domain }}
node_templates:
myApp:
type: MyApplication
artifacts:
image:
type: tosca.artifacts.Deployment.Image.Container.Docker
file: myapp:latest
repository: docker_hub
requirements:
- host: compute
- db:
node: mydb
relationship: mydb_connection
mydb:
type: tosca.nodes.Database
properties:
name: mydb
requirements:
- host: compute
compute:
type: tosca.nodes.Compute
capabilities:
host:
properties:
num_cpus: 1
disk_size: 200GB
mem_size: 512MB
relationship_templates:
mydb_connection:
type: DatabaseConnection
properties:
username: myapp
password:
eval:
secret: myapp_db_pw
# Generated by tosca.yaml2python from docs/examples/service-template.yaml at 2024-07-26T13:42:20 overwrite not modified (change to "overwrite ok" to allow)
"""An illustrative TOSCA service template"""
import unfurl
from typing import Sequence, Union
import tosca
from tosca import Attribute, Eval, GB, MB, Property, TopologyInputs, TopologyOutputs
import unfurl.configurators.shell
from unfurl.tosca_plugins.expr import concat
class Inputs(TopologyInputs):
domain: str
class Outputs(TopologyOutputs):
url: str = concat("https://", Inputs.domain, "/api/events")
class DatabaseConnection(tosca.relationships.ConnectsTo):
username: str
password: str = Property(metadata={"sensitive": True})
class MyApplication(tosca.nodes.SoftwareComponent):
domain: str = Inputs.domain
private_address: str = Attribute()
host: Sequence[
"tosca.relationships.HostedOn | tosca.nodes.Compute | tosca.capabilities.Compute"
] = ()
db: "DatabaseConnection"
def create(self, **kw):
return unfurl.configurators.shell.ShellConfigurator(
command=["create.sh"],
)
def configure(self, **kw):
return unfurl.configurators.shell.ShellConfigurator(
command=["configure.sh"],
)
def delete(self, **kw):
return unfurl.configurators.shell.ShellConfigurator(
command=["delete.sh"],
)
compute = tosca.nodes.Compute(
"compute",
host=tosca.capabilities.Compute(
num_cpus=1,
disk_size=200 * GB,
mem_size=512 * MB,
),
)
mydb = tosca.nodes.Database(
"mydb",
name="mydb",
host=[compute],
)
mydb_connection = DatabaseConnection(
"mydb_connection",
username="myapp",
password=Eval({"eval": {"secret": "myapp_db_pw"}}),
)
myApp = MyApplication(
"myApp",
host=[compute],
db=mydb_connection[mydb],
)
myApp.image = tosca.artifacts.DeploymentImageContainerDocker(
"image",
file="myapp:latest",
repository="docker_hub",
)