top of page
Writer's pictureLuispe Toloy

IDP - Why, when and how

To start I would like to talk about the acronym IDP .


A few years ago, when I started working full-time in the “infrastructure area”, we were talking about Internal Developer Platform and this refers to the infrastructure and set of tools that automate and facilitate the development, deployment and operation of the underlying services and resources, it has a focus on automation and integration of all the layers that a product development team needs, it includes CI/CD components, IaC, container orchestration, observability, etc.


Over the years we started talking about Internal Developer Portal , which is an evolution of an Internal Developer Platform.


According to Gartner, “By 2026, 80% of large software engineering organizations will establish platform engineering teams as internal providers of reusable services, components, and tools for application delivery. Platform engineering will ultimately solve the core problem of cooperation between software developers and operators.”


With Internal Developer Portal we begin to have a product approach as far as the platform is concerned, it may or may not include a UI and it facilitates self-service tasks with operation flows designed as a product and based on the internal development and security needs that the company requires.


So as not to bore you, I would like you to remember the following concepts:

  • Self-service

  • Standardization

  • Security

  • Reduce cognitive load

  • Platform as a product


Because an IDP

Having made the very brief introduction, let's talk about why.


Akua is, as is public knowledge, “The acquiring processing platform for all payment rails in Latam” and developing services in this context requires special care in security without losing speed in development and deployment and having autonomy for the operation/management of the services and associated resources.


With this context, we formed the Platform team at Akua to develop the IDP, with an aggressive but motivating challenge ahead, to develop the IDP from minute zero so that the product development team can deploy and manage microservices and the associated resources, all without losing sight of the concepts I mentioned previously, Self-service, Standardization, Security, Reduce cognitive load, Platform as a product .


Was it a whim? No, I have known Juanjo for a long time and we know from shared experience that having an IDP in a company that needs to be PCI compliant, following the highest levels of security with high availability and scalability is a very, very valuable benefit.


At Akua we could have created everything by hand, without pursuing standards, but it is a temporary solution. Let us consider that for some reason the industry created the concept of IaC, not just on a whim, but rather a need that arose almost naturally due to the implications and challenges in managing the infrastructure.


When an IDP

I anticipated something but we did it from minute zero and it was challenging, very challenging to think about:

  • What IaC tools are we going to use?

  • What are we going to use for CI/CD?

  • How do we keep our IDP from becoming a monster and keep it simple and easy to manage and scale?

  • Where will Akua's services be run?

  • How secure are we by default?


And a question may appear in the air: why from such an early stage?


Akua needs to have a standardized and secure development speed and maintain it over time, and that is very difficult to achieve without an IDP or it is not possible. You could “get out quickly” without having an IDP, but I assure you that sooner rather than later everything becomes chaos, for the product and for the management and maintenance of the infrastructure and software.

As

Let's address the last point, the how.


There are no recipes, sorry, but if you came looking for one, this post is not for you 😂

What I will tell you is what our IDP currently offers.


But before talking about what it offers, I'm going to tell you what we use to offer what we offer:

  • As an IaC tool we use Pulumi with Typescript.

  • For CI/CD we use Gitlab with private runners to interact with our cloud provider.

  • Speaking of cloud provider we use AWS (Amazon Web Services)

  • For application configuration we use Helm and develop a chart that all microservices use.

  • Where do our applications run? in kubernetes

  • For Observability we use NewRelic



Ok Luispi, so what?


I previously talked about cognitive load and the desire to keep our IDP simple and maintainable over time.

Well, to meet that objective we developed an internal library (as I mentioned with Pulumi and Typescript) that contains the business and security logic for the creation, removal and modification of platform resources.


To avoid our product development team having to learn an IaC tool, and to ensure that the Platform team has the freedom to change course if we need to, management is done from Port (https://www.getport.io/), where each team responsible for the application can create, update and delete resources with a simple form. Don't you believe me? Here's a gif.

When the application is synchronized, under the hood we run a private Gitlab runner on AWS and, relying on our IaC tool (Pulumi), we validate the status of what we want to deploy with what exists in the cloud.


Beneath this “simple flow” there are many hours of design and development that we had and continue to have with the Platform team. At the time of writing this post, from our IDP you can manage:

  • As I mentioned microservices in Kubernetes and this includes: deployment, service account, service, ingress, stateful set, cron job, HPA, light cache (?)

  • SQS

  • SNS

  • S3

  • DynamoDB

  • RDS Aurora

  • IAM Role, Security Group, Secret and ECR for the application/microservice

  • Multiple environments

  • Single and multi-tenant


All these resources taking into account cost optimization and security.


Before continuing I want to talk about how the last two items on the list work, Multiple environments, Single and multi tenant .


Multiple environments

When we talk about multiple environments, we mean that an application can run in different environments throughout its life cycle until it reaches production.


The approach and solution that we provided from our IDP in Akua is to abstract the creation of resources environment by environment. We went to a more abstract concept where we create or model the resources that the service will need to solve a business problem and then be able to select in which environment we want to synchronize the previously defined resources, being able to choose between a test and production environment.

Here is a short gif:


Single and multi tenants

Our Platform and our portal are designed and developed to support single/multi tenant, and not only at the product software level, but also throughout the entire technological stack that our platform has, and by this I mean:

  • Application Workloads

  • Resources from our cloud provider

  • Observability (tracing, logs, metrics)

  • etc


And this functionality was developed respecting one of the mantras that should be pursued when creating an IDP : Reduce cognitive load , why? Our product technology team can configure whether the application will run in 1..n tenants and when deploying can select environment and tenant, the rest is taken care of by our IDP to understand how, when and where to configure all the underlying resources 😎.


Conclusions

Throughout the publication we try to address in a more or less abstract way everything that surrounds an IDP, from what the acronyms mean to how a Platform team and an IDP can impact your company's business, in our case at Akua 🫶🏼.


Some things I would like to summarize what an IDP can offer you are:

 

💸 Cost reduction: By optimizing infrastructure and automating processes, organizations can achieve significant savings in operational and maintenance costs.


⚙️ Greater agility and speed in delivery: The implementation of a robust and flexible platform allows for faster time to market for new applications and functionalities, providing a competitive advantage in an ever-evolving market.


📈 Improved scalability and availability: Microservices-based architecture and container orchestration allow for efficient scaling of platform resources, ensuring high availability and performance.


🛡 Improved security and compliance: Applying security practices and using advanced tools helps protect sensitive data and comply with security and privacy standards.

 

Are decisions unilateral? Of course not, to build an IDP if the needs of “customers” are not listened to, everything can be an uphill battle.


There is still a long way to go, but we are very proud of what we have built, are building and will build at the IDP.


Until next time 👋🏼

9 views0 comments

Recent Posts

See All

Comments


bottom of page