Why we chose Pulumi

Navigating the vast world of cloud computing can often feel like trying to piece together a jigsaw puzzle with a myriad of complex pieces.

The Nitric framework provides a set of APIs and foundational cloud resources such as queues, buckets, schedules, and events. These tools aim to assist developers in building resilient and scalable applications without needing to navigate each cloud service.

A key feature of the framework is its ability to analyze application code. This analysis results in the automatic generation of specification that details the necessary resources and policies for the application's operation. The goal is to simplify the development process and to align resource allocation more closely with the application's actual needs.

Our team looks for tools that not only fit our immediate needs but align with our broader company culture and long-term vision. In the case of provisioning infrastructure, we chose Pulumi as our first IaC tool. Nitric uses the Pulumi Automation API for its pre-built provider plugins.

Here are the top reasons why we chose to implement our initial infrastructure providers with Pulumi.

An Alignment with the DevOps Culture

DevOps, at its core, seeks to break down the barriers between development and operations. It emphasizes collaboration, automation, and a shared understanding between the two traditionally siloed teams. Working with programming languages instead of low code or configuration creates another point of shared understanding between development and operations.

Speaking the same language

Pulumi allows teams to define infrastructure using general-purpose programming languages like Python, TypeScript, Go, and C#. This means that developers and operations can use a programming language that their team is already familiar with for their infrastructure code. We did experiment with providers written using CDKTF, but, found Pulumi providers to have greater usage, support, and stability to get us started.

Note: Developers of providers with Nitric are free to use any IaC solution including Terraform.

For the providers built for our open-source framework, we opted for the performance of GO. Using the Pulumi automation engine and GO, our team could avoid several challenges that we'll illustrate below.

Modularity by Default

With modularity as a goal for Nitric’s developer experience, it was a key technological evaluation for our infrastructure provisioning.

Modularity Challenges with Terraform

While building simple components with Terraform can be fairly straightforward, once you get into reuse and modularity there is a lot of repetition of boilerplate code and string substitution.

Let’s illustrate this with a basic description of a cloud-watch event rule in AWS for a single cloud-watch event.

resource "aws_cloudwatch_event_rule" "every_five_minutes" {
  name                = "every-five-minutes"
  description         = "Fires every five minutes"
  schedule_expression = "rate(5 minutes)"
}

To convert this into a reusable module, we’ll need to separate the configurable parts into input variables and potentially also provide outputs, depending on the requirements.

variable "name" {
  description = "The name of the CloudWatch event rule."
  type        = string
  default     = "every-five-minutes"
}

variable "description" {
  description = "The description for the CloudWatch event rule."
  type        = string
  default     = "Fires every five minutes"
}

variable "schedule_expression" {
  description = "The scheduling expression for the CloudWatch event rule."
  type        = string
  default     = "rate(5 minutes)"
}

resource "aws_cloudwatch_event_rule" "schedule" {
  name                = var.name
  description         = var.description
  schedule_expression = var.schedule_expression
}

output "rule_arn" {
  description = "The Amazon Resource Name (ARN) of the rule."
  value       = aws_cloudwatch_event_rule.schedule.arn

Once we have our module, we can use this within our main infrastructure code by copying and configuring the following snippet.

module "my_schedule" {
  source              = "./cloudwatch_schedule_module"
  name                = "my-custom-name"
  description         = "My custom description"
  schedule_expression = "rate(10 minutes)"
}

output "my_schedule_rule_arn" {
  value = module.my_schedule.rule_arn
}

The limitations of this approach:

1. Control logic: It lacks the rich logic and control structures (e.g. loops, conditionals) that are present in general-purpose programming languages.

   1. Though recent versions of Terraform have introduced constructs like for_each, general-purpose languages have a broader intuitive set of iteration capabilities making them more appropriate for complex conditions.

2. Error handling: This was a big limitation for our goals. All errors in Terraform are runtime errors. Much of the inspection is done after drafting the configuration with inspection tools. Handling errors is straightforward in languages that offer strict typing, exception handling, or detailed error-handling constructs.

3. Encryption of secrets: Terraform stores its state in plaintext by default. If secrets are managed as regular state items in Terraform, they will be in plaintext in the state file. This is a well-known issue and the general guidance has been to use state backends that encrypt at rest, such as AWS S3 with server-side encryption enabled. Secrets stored in the Pulumi state are never stored in plain text. By default they are encrypted at rest and during transmission and can only be accessed with the encryption key.

4. Advanced testing: Testing infrastructure code in Terraform isn't as straightforward as unit testing in general-purpose languages. While there are tools like terratest that allow for this, they require learning and integration. Importantly, it is difficult to replicate this style of testing across a variety of projects or project teams.

   Pulumi’s support for standard programming languages means that testing is more integrated. You can employ the same rigorous testing methodologies to your infrastructure as you normally would with your software written in node, java, go, etc.

How Nitric Leverages Pulumi for Modularity

Pulumi infrastructure components are modular by default because they are easily composed into classes or functions which promotes reusability across different projects.

  // Create a new eventbridge schedule
  res.Schedule, err = scheduler.NewSchedule(ctx, name, &scheduler.ScheduleArgs{
    ScheduleExpression:         pulumi.String(awsCronValue),
    ScheduleExpressionTimezone: pulumi.String(args.Tz),
    FlexibleTimeWindow: &scheduler.ScheduleFlexibleTimeWindowArgs{
      Mode: pulumi.String("OFF"),
    },
    Target: &scheduler.ScheduleTargetArgs{
      Arn:     args.Exec.Function.Arn,
      RoleArn: role.Arn,
      Input:   pulumi.Sprintf(scheduleInputTemplate, name),
    },
  })

Nitric’s goal is to elevate the developer experience by applying principles like DRY (Don’t repeat yourself). Each time an application is deployed, Nitric inspects the code to build a resource map. In the following example, our resource map will result in a scheduled task and the associated function. During deployment this resource map is fulfilled by providers that associate the requests with appropriate resources.

// Run every 5 minutes
schedule('process-transactions').every('5 minutes', async (ctx) => {
  console.log(`processing at ${new Date().toLocaleString()}`)
})

Pulumi's inherent modularity in its infrastructure components offers a robust foundation for efficient code composition and reusability across projects. By leveraging Pulumi, Nitric not only streamlines the developer experience but also aligns operational governance and standardization practices making it a great strategic choice for infrastructure automation.

Open Source Licensing

Terraform's choice of the Business Source License 1.1 might not strike the right chord with everyone. This sentiment is underscored by support for initiatives like Open-Tofu.

Pulumi’s decision to go with the Apache License 2.0 and its commitment to fostering a free, open community aligns well with Nitric’s open source approach and values.

We believe that being a part of an open-source community enriches developer experiences by fostering collaborative learning and innovation. This collective effort drives the simplification of building applications in the cloud, making technology more accessible and efficient for all.

In Summary

While we aim to allow users of the Nitric framework to work with their Infrastructure as Code tool of choice, we also took an analytical and philosophical approach to selecting the solution to guide our initial providers.

Pulumi's alignment with DevOps culture, modularity for reusability, advanced error handling, and commitment to open-source principles made it a great choice for us. We're confident our use of Pulumi will help our infrastructure remains agile, robust, and in tune with the needs of our development teams.