Introduction🍃

Managing cloud infrastructure efficiently is not only essential but also critical in today’s fast-paced tech environment. Traditional approaches like ClickOps and Infrastructure as Code (IaC) are commonly used, but each comes with its own sets of challenges. System Initiative offers a fresh approach to cloud infrastructure management by combining dynamic modeling, real-time collaboration, and DevOps principles. In this blog, we’ll explore the challenges of ClickOps and IaC and demonstrate how SI can address these issues, providing a smoother and more adaptable solution.

What is System Initiative?

System Initiative (SI) was launched in 2019 with the goal of addressing the limitations of traditional IaC tools, like Terraform. While these conventional tools have been effective in automating infrastructure provisioning, they often struggle to manage the complexity and dynamic nature of modern cloud environments. SI aims to solve these challenges by combining DevOps principles with automation, dynamic modeling, and real-time collaboration and feedback mechanisms.

Moreover, SI offers a user-friendly interface for visualizing, simulating, and controlling infrastructure. A key advantage is its built-in collaborative workspace, enabling multiple users to work together by default. This not only streamlines complex system management but also fosters greater collaboration between development and operations teams. The platform provides a real-time feedback loop, ensuring that changes can be evaluated and optimized on the fly. As a result, SI helps teams adapt quickly to changing demands, optimize infrastructure efficiency, and reduce the chances of failed deployments.

In addition to eliminating the silos between teams and providing a collaborative platform, SI integrates seamlessly with AWS, utilizing its scalability and flexibility, and is continuously expanding its capabilities to support other cloud providers like Azure and Google Cloud.

Challenges with ClickOps & Traditional IaC

Limitations of ClickOps

ClickOps refers to the manual configuration of cloud infrastructure through graphical user interfaces (GUI), such as the AWS Console. While it may seem like an intuitive and straightforward way to manage resources, ClickOps introduces several challenges when it comes to scaling, automating, and maintaining consistency across environments:

Lack of Version Control: Manual changes made through the GUI cannot easily be tracked or version-controlled. This makes it difficult to audit, revert, or ensure consistent versions across different environments.

Automation Limitations: With manual configurations, it is difficult to automate the setups especially for mid-scale to large-scale infrastructure deployments. Without automation, ensuring consistent configurations across multiple environments can be a challenge.

Scalability Issues: Manual click-based operations can lead to inefficiencies and potential errors when the size of infrastructures increase. Configuring resources one by one across multiple interfaces not only takes more time, but it can be more prone to the risk of human errors.

Limitations of Traditional IaC

To overcome some of the challenges posed by ClickOps in infrastructure management, many choose to use IaC as a more efficient and scalable alternative. While IaC offers significant advantages in automation and consistency, it also introduces its own set of challenges:

State Management Issues: IaC tools (e.g. Terraform) uses state files to keep track of the infrastructure. These state files can easily fall out of sync with the actual resources, especially when manual changes are made directly in the AWS console.

Delayed Feedback Loops: Deployments with IaC typically involves waiting for changes to be applied before being able to assess the changes/impact. This could slow down iteration cycles and make troubleshooting more inefficient.

Large-Scale Deployment Complexity: Dependencies, module versions, and multiple environments could create unstable systems that are difficult to maintain when the infrastructure scales and becomes increasingly complex.

ClickOps v.s. IaC v.s. System Initiative

While traditional IaC, like Terraform and Pulumi, offers significant improvements over manual ClickOps workflows, SI goes further by enhancing aspects such as automation, collaboration, and feedback loops in a way that redefines how infrastructure management can be executed.

We have summarized in the table above how these aspects vary across ClickOps, traditional IaC, and SI. In the next few sections, we will dive deeper primarily into how SI addresses each of these aspects, such as its real-time collaboration, dynamic modeling, faster feedback loops, and flexible customization, and demonstrate how SI represents the next evolution of infrastructure management.

How SI Solves ClickOps/Traditional IaC Challenges

Real-Time Collaboration

SI offers its real-time collaboration feature by default, allowing multiple users to work together seamlessly on the same workspace. This collaborative approach enhances efficiency and accuracy in managing infrastructure. Any changes made can be applied in AWS by simply merging the branched version of the Change Set to the HEAD change set.

If any changes are made, If an approval flow is needed, teams can instantly verify the change and simulate in AWS any resource changes, ensuring system-wide alignment while minimizing the risk of errors. For users working on the same change set, they can vote on whether or not the change should be merged to the HEAD change set.

Hypergraph of Reactive Functions

System Initiative leverages a hypergraph of reactive functions to simulate dynamic infrastructure changes and dependencies in real-time. Imagine a hypergraph as a large network made up of nodes (representing assets) and edges (representing the connections between them). When a new change is proposed, it’s treated as another node in the hypergraph, connected to its own relevant assets. This structure allows SI to track the current state of the system, its linked assets, and how potential changes might affect the overall infrastructure.

In addition, when a resource is changed, all related resources automatically update to reflect this change. For example, if an AWS resource such as an instance or network is modified, the system dynamically adjusts the status of all linked assets, ensuring real-time visibility, which then enables smarter, more informed decision-making.

The figure above is a graphical representation showing how each resource within System Initiative (SI) is interconnected through nodes and edges, forming a hypergraph structure.

Dynamic Modeling Instead of State Files

Unlike traditional state files, which capture a snapshot of infrastructure at a single point in time, SI uses dynamic modeling to maintain a real-time view of system configurations and dependencies. This “HEAD set” represents a live snapshot of your AWS infrastructure as configured in SI. However, it’s important to note that only changes made within SI are reflected in real-time; changes made directly in the AWS Console aren’t automatically synced. While bi-directional synchronization isn’t fully implemented yet, there is a “refresh” capability that allows manual syncing of the SI system to align with the current status of AWS services. Regardless, the dynamic modeling greatly enhances visibility into infrastructure changes within the platform.

Fast Feedback Loops

One of the standout features of SI is its fast feedback loops. Changes made within the platform are immediately reflected, offering real-time insights without the usual wait times associated with infrastructure provisioning. This rapid feedback enables users to quickly assess and optimize configurations, improving both efficiency and accuracy.

Digital Twins for Infrastructure Modeling

SI introduces Digital Twins, which allow users to create virtual representations of AWS environments. These “twins”, enabled by hypergraph, simulate what the actual AWS environment would look like once changes are applied. This capability provides a risk-free space for testing and optimizing infrastructure changes, ensuring that modifications can be thoroughly vetted before impacting production systems.

Customization through TypeScript Functions

In System Initiative, users can write TypeScript functions for complete customization and control over infrastructure definitions. Users can leverage the AWS SDK for JavaScript-Node.js to directly interact with AWS resources, and they can also utilize any JavaScript libraries. For any node.js packages that are not yet integrated into SI, you can easily request the SI team to add those custom packages to their virtual machine that hosts SI. For simpler use cases, like interacting with third-party API endpoints, you can use JavaScript fetch functions to code any custom interactions, providing immense flexibility for a wide range of use cases.

System Initiative vs Other IaC Tools

This comparison highlights key differences between System Initiative, Pulumi, and Terraform, focusing on their approaches to IaC and collaboration features.

SI is an open-source platform that offers multi-user collaboration and faster feedback through its ability to simulate and provision infrastructure quickly, while also providing flexibility with TypeScript for custom functions. However, it may not scale as easily.

In contrast, Pulumi and Terraform—both requiring paid versions for collaboration—operate with slower feedback loops, meaning users may experience a delay in identifying infrastructure configuration issues. Pulumi supports multiple languages, including TypeScript, Python, Go, and C#, while Terraform uses its own declarative language, HCL. Both Pulumi and Terraform excel at replicating infrastructure once the code is established, though Terraform is no longer open-source, unlike SI and Pulumi.

Real-world Use Case

Our team created a demo with the objective to incorporate SI in building an email signature generator for Megazone global employees. It is a static webpage that aims to facilitate the email signature generation and distribution while leveraging AWS infrastructure. The overall architecture is shown in Figure 5.

The website will be hosted on S3 for serving static content, with CloudFront used as a Content Delivery Network to distribute the website globally.Employees will sign in using Google SSO, which is restricted to the domains @mz.co.kr and @megazone.com. After sign-in, users will fill out an email signature content form to provide necessary information for their signature, as shown in Figure 6.

Getting Started with System Initiative

Please refer to this link for a quick guide on how to get started on SI:

https://docs.systeminit.com/tutorials/getting-started 

Current Status, Pain Points, and Future Improvements

As mentioned in the previous section, our team had the opportunity to collaborate closely with the System Initiative team while working on a project to host a static web page on AWS. Despite being a small startup with around 17 employees, the SI team’s enthusiasm and dedication were remarkable. They were highly responsive, providing guidance, advice, and support throughout the entire process. Although our project was relatively small, they exceeded our expectations by scheduling multiple meetings, answering all of our questions, and walking us through the necessary steps. Additionally, they updated their features and documentation to include the specific functions we needed, demonstrating their strong commitment to customer success.

Here are some key observations that we noted while creating our demo:

S3 Features

• Initially, SI only included the “Create Bucket” feature for S3.
• After we consulted with the SI team, new features such as “Bucket Website,” “Bucket Policy,” and “Bucket Public Access Block” became available. 
• In addition, the Bucket asset has a feature called “Acl”, but that can only be set as private, regardless of what other options are available.

Route 53 Assets (Domain registration / hosted zone)

• The Domain Registration asset does not yet exist in SI for Route 53. When a domain is created via Route 53, a hosted zone for that domain is automatically generated. However, this AWS feature makes SI’s hosted zone asset less useful in this case, as it does not sync with the automatically created hosted zone, which comes with domain registration by default on AWS.
• If you look carefully at Figure 8, you will see that the hosted zone does not exist, as it is set up manually.

Cloudfront cannot be deleted once created on SI

• Cloudfront needs to be manually disabled via console before SI can delete it. Sometimes even when it is disabled, it cannot be deleted. 

Bi-directional Real-time Sync is not perfected

• Only the changes made within SI will be represented in real-time.
• Changes made directly in the AWS Console won’t be reflected to SI automatically.
  • There is a “Refresh” button in SI that is used to sync and reflect the changes made from the AWS Console – this function is not fully functional yet.
  • Resources created within AWS console cannot be imported into SI – discovering/importing existing AWS resources is on their roadmap.
• Thus, the Dynamic Modeling “state files” might not maintain a real-time view of the actual AWS system configurations and dependencies.

Version Control Reversioning

• If a Change Set is accidentally merged to the Head set, there is no method in place to revert back to the version before the accidental changes were made.
• As of this moment, there is no way to check which user made what changes in the history.

User Permissions and Roles

• Currently there are 2 roles a member can have: Editor or Owner.
  • Editors have 1 less privilege than Owners, which is to Delete the Workspace.
  • However, an Editor can grant themselves to be an Owner.
  • There is no Safeguard for Delete Workspace Confirmation
• There is no Read-only permission (or any other permission) to limit workspace users from certain acts.
  • Anyone invited to the SI workspace

Conclusion🍃

System Initiative offers a fresh and innovative approach to cloud infrastructure management, tackling many of the limitations found in ClickOps and traditional IaC. Through its dynamic modeling, real-time collaboration, and fast feedback loops, SI empowers teams to work more efficiently, reduce the risk of errors, and enhance overall infrastructure visibility. Although the platform is still maturing—particularly in areas like bi-directional sync and version control reversioning—its ability to simplify complex cloud deployments makes it a strong contender in the infrastructure management space. As SI continues to evolve, the commitment and responsiveness of its team provide a promising future for addressing user pain points and expanding its functionality. For organizations seeking more dynamic and adaptable cloud management solutions, SI represents a powerful tool that is well worth exploring.

Reference

Jacob, A. (2024, September 25). System initiative is the future. System Initiative.
https://www.systeminit.com/blog/system-initiative-is-the-future 

Stack, P. (2023, June 27). My journey to system initiative. System Initiative.
https://www.systeminit.com/blog/my-journey-to-system-initiative 

Jacob, A. (2023, June 21). Second Wave devops. System Initiative.
https://www.systeminit.com/blog/second-wave-devops 

Jacob, A. (2023a, June 20). Five breakthroughs on the path to system initiative. System Initiative.
https://www.systeminit.com/blog/five-breakthroughs 

Jacob, A. (2023a, February 4). DevOps without papercuts. System Initiative.
https://www.systeminit.com/blog/devops-without-papercuts 

kvgru. (2022). R/devops on reddit: I think IAC is a lot better than “ClickOps”! Redit. 

https://www.reddit.com/r/devops/comments/znfctz/i_think_iac_is_a_lot_better_than_c

lickops/ 

Shaughnessy, L. (2023, September 22). Infrastructure as code is not the answer!. Medium. 

https://lukeshaughnessy.medium.com/infrastructure-as-code-is-not-the-answer-cfaf4882dcba

Ahmed, D. (n.d.). Do not click-ops your data infrastructure. RSS. 

https://platformengineering.org/blog/data-infrastructure-do-not-click-ops 

Written by 

• David Jeong, Project Manager, MegazoneCloud US
• Heeseung Hong, Cloud Engineer Intern, MegazoneCloud Canada
• Shin Taniguchi, Cloud Engineer Intern, MegazoneCloud Canada