Paco configuration overview¶
Paco configuration is a complete declarative description of a cloud project. These files semantically describe cloud resources and logical groupings of those resources. The contents of these files describe accounts, networks, environments, applications, resources, services, and monitoring configuration.
The Paco configuration files are parsed into a Python object model by the library
paco.models. This object model is used by Paco to provision
AWS resources using CloudFormation. However, the object model is a standalone
Python package and can be used to work with cloud infrastructure semantically
with other tooling.
File format overview¶
Paco configuration is a directory of files and sub-directories that make up an Paco project. All of the files are in YAML format.
In the top-level directory are sub-directories that contain YAML files each with a different format. This directories are:
accounts/: Each file in this directory is an AWS account.
netenv/: Each file in this directory defines a complete set of networks, applications and environments. Environments are provisioned into your accounts.
monitor/: These contain alarm and logging configuration.
resource/: For global resources, such as S3 Buckets, IAM Users, EC2 Keypairs.
service/: For extension plug-ins.
Also at the top level are
paco-project-version.txt is a simple one line file with the version of the Paco project
file format, e.g.
2.1. The Paco project file format version contains a major and a medium
version. The major version indicates backwards incompatable changes, while the medium
version indicates additions of new object types and fields.
project.yaml contains gloabl information about the Paco project. It also contains
paco_project_version field that is loaded from
The YAML files are organized as nested key-value dictionaries. In each sub-directory, key names map to relevant Paco schemas. An Paco schema is a set of fields that describe the field name, type and constraints.
An example of how this hierarchy looks, in a NetworksEnvironent file, a key name
must have attributes that match the Network schema. Within the Network schema there must be
an attribute named
vpc: which contains attributes for the VPC schema. That looks like this:
network: enabled: true region: us-west-2 availability_zones: 2 vpc: enable_dns_hostnames: true enable_dns_support: true enable_internet_gateway: true
Some key names map to Paco schemas that are containers. For containers, every key must contain
a set of key/value pairs that map to the Paco schema that container is for.
Every Paco schema in a container has a special
name attribute, this attribute is derived
from the key name used in the container.
For example, the NetworkEnvironments has a key name
environments: that maps
to an Environments container object. Environments containers contain Environment objects.
environments: dev: title: Development staging: title: Staging prod: title: Production
When this is parsed, there would be three Environment objects:
Environment: name: dev title: Development Environment: name: staging title: Staging Environment: name: prod title: Production
Key naming warning: As the key names you choose will be used in the names of resources provisioned in AWS, they should be as short and simple as possible. If you wanted rename keys, you need to first delete all of your AWS resources under their old key names, then recreate them with their new name. Try to give everything short, reasonable names.
Key names have the following restrictions:
- Can contain only letters, numbers, hyphens and underscores.
- First character must be a letter.
- Cannot end with a hyphen or contain two consecutive hyphens.
Certain AWS resources have additional naming limitations, namely S3 bucket names can not contain uppercase letters and certain resources have a name length of 64 characters.
title field is available in almost all Paco schemas. This is intended to be
a human readable name. This field can contain any character except newline.
title field can also be added as a Tag to resources, so any characters
beyond 255 characters would be truncated.
Many Paco schemas have an
enabled: field. If an Environment, Application or Resource field
enabled: True, that indicates it should be provisioned. If
enabled: False is set,
then the resource won’t be provisioned.
To determine if a resource should be provisioned or not, if any field higher in the tree
is set to
enabled: False the resource will not be provisioned.
In the following example, the network is enabled by default. The dev environment is enabled, and there are two applications, but only one of them is enabled. The production environment has two applications enabled, but they will not be provisioned as enabled is off for the entire environment.
network: enabled: true environments: dev: enabled: true default: applications: my-paco-example: enabled: false reporting-app: enabled: true prod: enabled: false default: applications: my-paco-example: enabled: true reporting-app: enabled: true
Note that currently, this field is only applied during the
paco provision command.
If you want delete an environment or application, you need to do so explicitly with the
paco delete command.
References and Substitutions¶
Some values can be special references. These will allow you to reference other values in your Paco Configuration.
paco.ref netenv: NetworkEnvironment reference
paco.ref resource: Resource reference
paco.ref accounts: Account reference
paco.ref function: Function reference
paco.ref service: Service reference
References are in the format:
In addition, the
paco.sub string indicates a substitution.
To refer to a value in a NetworkEnvironment use an
paco.ref netenv reference. For example:
paco.ref netenv should be a part which matches the filename of a file (without the .yaml or .yml extension)
in the NetworkEnvironments directory.
The next part will start to walk down the YAML tree in the specified file. You can
either refer to a part in the
Keep walking down the tree, until you reach the name of a field. This final part is sometimes a field name that you don’t supply in your configuration, and is instead can be generated by the Paco Engine after it has provisioned the resource in AWS.
An example where a
paco.ref netenv refers to the id of a SecurityGroup:
network: vpc: security_groups: app: lb: egress webapp: ingress: - from_port: 80 name: HTTP protocol: tcp source_security_group: paco.ref netenv.my-paco-example.network.vpc.security_groups.app.lb
You can refer to an S3 Bucket and it will return the ARN of the bucket:
artifacts_bucket: paco.ref netenv.my-paco-example.applications.app.groups.cicd.resources.cpbd_s3
SSL Certificates can be added to a load balancer. If a reference needs to look-up the name or id of an AWS
Resource, it needs to first be provisioned, the
order field controls the order in which resources
are created. In the example below, the ACM cert is first created, then an Applicatin Load Balancer is provisioned
and configured with the ACM cert:
applications: app: groups: site: cert: type: ACM order: 1 domain_name: example.com subject_alternative_names: - '*.example.com' alb: type: LBApplication order: 2 listeners: - port: 80 protocol: HTTP redirect: port: 443 protocol: HTTPS - port: 443 protocol: HTTPS ssl_certificates: - paco.ref netenv.my-paco-example.applications.app.groups.site.resources.cert
To refer to a global resource created in the Resources directory, use an
paco.ref resource. For example:
paco.ref resource the next part should matche the filename of a file
(without the .yaml or .yml extension) in the Resources directory.
Subsequent parts will walk down the YAML in that file.
In the example below, the
hosted_zone of a Route53 record is looked up.
# netenv/my-paco-example.yaml applications: app: groups: site: alb: dns: - hosted_zone: paco.ref resource.route53.example # resource/Route53.yaml hosted_zones: example: enabled: true domain_name: example.com account: paco.ref accounts.prod
To refer to an AWS Account in the Accounts directory, use
paco.ref. For example:
Account references should matches the filename of a file (without the .yaml or .yml extension) in the Accounts directory.
These are useful to override in the environments section in a NetworkEnvironment file to control which account an environment should be deployed to:
environments: dev: network: aws_account: paco.ref accounts.dev
A reference to an imperatively generated value that is dynamically resolved at runtime. For example:
This must be an importable Python function that accepts three arguements: reference, project, account_ctx.
This function must return a value that is compatable with the fields data type (e.g. typically a string).
There is one built-in function:
Currently can only look-up AMI IDs. Can be either
web: type: ASG instance_ami: paco.ref function.aws.ec2.ami.latest.amazon-linux-2
To refer to a service created in the Services directory, use an
paco.ref service. For example:
Services are plug-ins that extend Paco with additional functionality. For example, custom notification, patching, back-ups and cost optimization services could be developed and installed into an Paco application to provide custom business functionality.
Can be used to look-up a value and substitute the results into a templated string.