Infrastructure as Code With Pulumi

Why Pulumi?

For defining and deploying necessary cloud infrastructure for your project applications, by default, Webiny relies on Pulumi, a modern infrastructure as code (IaC) solution.

Webiny chose Pulumi as its default IaC solution for a couple of reasons:

• cloud infrastructure is defined via code (TypeScript)
• it supports multiple cloud providers (AWS, Azure, Google Cloud Platform, …)
• it provides multiple solutions for storing infrastructure state
• a vibrant and rising community

A Closer Look

In the following folder structure, we are looking at three applications that are set up by default in a new Webiny project. These are the API (./api), Admin Area (./apps/admin), and Website (./apps/website), which are shown in the following directory structure (code folder omitted for brevity):

Let’s examine how these applications are utilizing Pulumi in order to define and deploy needed cloud infrastructure resources.

Each of these applications are standalone Pulumi projects, which essentially means two things:

• every application contains its own infrastructure-related code (Pulumi program), configuration, meta, and state files
• every application is deployed separately (you cannot deploy two applications in a single operation)

It’s also important to know you can have multiple instances of your application’s cloud infrastructure up and running at the same time. This is achieved via Pulumi stacks:

Every Pulumi program is deployed to a stack. A stack is an isolated, independently configurable instance of a Pulumi program. Stacks are commonly used to denote different phases of development (such as development, staging and production) or feature branches (such as feature-x-dev, jane-feature-x-dev).

In the following section, we briefly cover the key files that form a Pulumi project, that can be found in your application folders. Note that if you want to dive deeper, you’re certainly encouraged to visit the official documentation which explains these concepts in more detail.

Project Files

The following are the key files of every Pulumi project.

Pulumi.yaml

The Pulumi.yaml project file specifies metadata about the Pulumi project. There can only be one project file per Pulumi project.

A Pulumi.yaml file might look something like the following:

One of the more important properties in this file is the backend property, which we cover in the following section.

Pulumi.stack-name.yaml

The Pulumi.{stack-name}.yaml file is a Pulumi stack settings file. There can be more than one stack settings files, for example, for a couple of different environments. For example, it’s possible to have Pulumi.dev.yaml, Pulumi.staging.yaml, Pulumi.production.yaml, and so on.

index.ts

This is the entry point file in which you start to define your infrastructure (the Pulumi program). You can organize the code in any way you like. For example, in the default API, Admin Area, and Public Website applications, the index.ts file just imports different classes, which encapsulate different cloud infrastructure segments. These classes can be found in the pulumi folder.

State Files

Except for the files we mentioned in the previous section, there is also one other group of files the Pulumi solution produces. These are Pulumi project’s state files.

These files describe at what state your cloud infrastructure is currently at. For example, it contains a list of all deployed resources, various metadata, configuration, and so on. State files are stored per Pulumi program stack, and by default, they are stored in the .pulumi folder, located in your project root.

Notice how state files are organized per project application:

• api - API project application’s cloud infrastructure state files
• apps/admin - Admin Area project application’s cloud infrastructure state files
• apps/website - Website project application’s cloud infrastructure state files

Different Backends

As mentioned, by default, Pulumi state files are stored in your project application folder, inside the .pulumi folder. While in some cases, storing them locally may work for you, in others, you may want to store these in a controlled, centralized, remote storage.

This is where the concept of Pulumi backends comes into play, which represent different storage services, that you can use for storing your state files.

For example, you can create a simple S3 bucket, and choose to store your files in it. You can also use the Pulumi Service, which, in addition to storing state files, also provides a couple of other cool features.

Read more about different backends in the following official documentation article.

FAQ

Are You Using Pulumi's Automation API?

Currently, Webiny is not using the Automation API. It’s actually using its own Pulumi SDK package, which is just a tiny wrapper over Pulumi CLI. Essentially, it enables programmatic use of the Pulumi CLI, which is similar to what the Automation API is also doing.

Is It Possible to Use a Different Infrastructure-as-Code Solution?

Switching to a different infrastructure-as-code (IaC) solution would require a significant amount of work as it would involve rewriting almost all of the Webiny IaC code from scratch for a different IaC solution. Therefore, we do not recommend it.

However, if you want to experiment with it, you would need to create a custom CLI plugin similar to the Webiny CLI plugin for Pulumi. This plugin is responsible for creating the deploy, destroy, watch, and other commands.
To use your custom plugin, you would then need to import it into the webiny.project.ts file and remove the Pulumi plugin. The most challenging part would be transforming all of the Pulumi code into respective IaC code (i.e. transforming all Pulumi code to CloudFormation, Terraform, or other IaC). You can refer to the Pulumi code to see the resources Webiny deploys and the Cloud Infrastructure documentation for more information.

Please keep in mind that this process will require a significant amount of effort, and it’s recommended to stick with Pulumi unless your organization has strict policies that require using a different IaC solution.

What Is Stack Output?

Stack output represents values that are exported from your Pulumi program. For example, at the end of your index.ts file (located in root of your project application folder), you might have something like the following:

Doing this can be useful if, for example, one, or more applications depend on these values.

Is It Possible to Read Stack Output From Another Stack?

It’s possible to read output from another stack. This is definitely useful if one or more of your current stack’s infrastructure resources are depending on one or more resources from another stack. To learn more, check out the Reading stack output article.

GettingCannot read property 'split' of undefinedError While Running Any Command

When running any command, such as yarn webiny info, if you are getting the following error:

Please specify an environment when running the command and it should work. For example: yarn webiny info --env dev. This bug has persisted for some time, preventing the command from working as expected when using a pulumi backend other than the local one.