Internal developer platform, IDP

We have probably heard many times about Customer experience (CX) or user experience (UX). For those of you who haven’t, both are very broad terms for the experience a person has. CX refers to the overall experience a customer has with a brand or organisation, including all touchpoints along their customer journey, while UX refers specifically to the experience a user has when interacting with a digital product or service, such as a website or app. Put another way, CX is a broader concept that encompasses the entire customer experience, while UX focuses on the user’s experience with a particular digital product or service.

But who is in charge of creating these digital products or services? Developers, through their designs and implementations, make the final solutions offer that unique user experience. And who is in charge of offering a unique experience to developers? We would then talk about developer experience (DevX or DX). To be able to offer a good experience to developers, we must (as in the previous cases), look at their journey; look at the steps and actions they take from the moment a solution is conceived until it is working in production.

Wouldn’t it be desirable for developers to focus on creating the desired solution without having to worry about configuring all the details of the deployment or without needing to know in detail all the technologies involved in its development?

This is where Internal Developer Platforms (IDP) come into play.

What is an IDP and who manages it?

There are many different types of platforms, but in this case, as the name suggests, it is aimed at programmers.The platform team is responsible for building the internal developer platform (IDP) in order to create golden paths and enable developer self-service. An IDP is composed of many different technologies and tools, brought together in a way that reduces the cognitive load on developers without abstracting from the underlying context and technologies. Following best practices, platform teams treat their platform as a product and build, maintain and continuously improve it with feedback from their users.

Noteworthy from the definition is the concept of a “platform team”, which is a grouping of other types of teams that provide an internal product (the platform) to accelerate delivery by teams aligned with the value stream. In other words, for optimal use of the platform, it is more than advisable to adapt the way of working, with teams focusing on different tasks (I recommend reading Team Topologies). But let’s not focus on the organizational part.

Team Topologies approach

Kaspar von Grünberg, Humanitec’s CEO defines an IDP “as a self-service layer that allows developers to interact independently with their organization’s delivery setup, enabling them to self-serve environments, deployments, databases, logs and anything else they need to run their applications.”

We can view the internal developer platform (IDP) as a collection of technologies and tools used by developers, properly linked, to minimize the cognitive load, streamline the process and improve the developer experience. The IDPs are very varied and cover functionalities such as version control, continuous integration and deployment (CI/CD) pipelines, container orchestration, infrastructure provisioning, automated testing and observability, among others. Likewise, this collection of technologies and tools can be of its own generation, free software or proprietary. Within the latter type we can name Qovery, OpsLevel, Humanitec, Argonaut, Gopaddle, or Portainer, although there are many more.

IDPs typically cover 5 major functions:

Application Configuration Management: to enable managing application configuration as code in a standardized, reliable, dynamic and scalable way. This has a significant impact on application configuration maintenance, debugging and governance.

Infrastructure orchestration: IDP is the sum of the technology and tools that an Ops, DevOps or platform engineering team brings together to build golden paths for developers. All existing tools and infrastructures are part of IDP, integrated with it and orchestrated by IDP to enable Continuous Delivery or even Continuous Deployment (CD) processes.

Environment Management: to allow developers to self-manage fully provisioned environments on demand. This eliminates many bottlenecks and enables faster delivery. Each new environment is provisioned as defined by the platform team.

Deployment management: to enable teams to move to a continuous deployment (CD) process. It also provides a clear record of each deployment performed, which is very useful for audits and similar processes.

Role-based access control: to manage access at a granular level. This can limit production access to a small number of trusted individuals, while allowing each engineer to create new development environments as needed.

In order to facilitate the use of IDPs, and depending on the maturity of the platforms, we can find several ways to access the functionalities offered by them, for example from the command line, through a graphical interface or even through portals.

The portals, known as Internal Developer Portal (oh no!, another IDP!, but we will use it only to refer to the platforms) are worth mentioning.
Gartner defines them as:

“Internal developer portals serve as the interface through which developers can discover and access internal developer platform capabilities.”
Gartner

Many internal developer platforms also offer portals for developers to use, sometimes blurring the line between portals and platforms.

Benefits of platforms

These platforms offer multiple key benefits not only for programmers, but for the total organization.

Standardization and automation

They provide a standardized set of self-service tools and services for developers, reducing fragmentation and complexity.

They can automate many manual tasks associated with development and operations, such as environment configuration, code deployment, and infrastructure management.

They automate recurring tasks such as configuring development environments, building pipelines and deploying applications, allowing developers to focus more on writing code.

Help standardize development and operations processes across the organization. This leads to improved code consistency and quality.

Collaboration and integration

By providing standardization, they facilitate cross-team collaboration and ease of integration between teams.

They foster collaboration between development, operations and security teams by providing a shared platform.

Accelerate onboarding of new developers by providing resources, documentation and best practices in one place.

Allow new teams to quickly integrate with the same tools and services.

Efficiency and scalability

Enable developers to focus on writing code rather than managing infrastructure. This speeds development time and enables teams to deliver products faster.

Thanks to standardization and collaboration capabilities, they facilitate scalability by allowing development teams to deploy and manage large-scale applications with less effort.

Governance and compliance

Establish a governance framework that allows for flexibility and adherence to best practices, while meeting security and compliance requirements.

By standardizing and automating development and operations processes, we can include control steps for security and compliance, improving the security of the code and the end infrastructure.

Dangers of platforms

The use of IDPs is not all advantages. In the book Platform Strategy, Gregor Hohpe details several of the unintended consequences of using these platforms.

For example, by introducing an abstraction layer between the programmer and the different tools, the programmer learns to use this abstraction layer and not the tools themselves. The programmer will find the job less attractive, because he will not grow in knowledge of the tools and it will be more difficult for him to find work in the future, since he will only know how to use this internal tool.

Another danger discussed in the book is that the tools will evolve, and the IDP must evolve with them to be able to offer these new functionalities. In this scenario, there are two possible options: to maintain and update the platform, which implies an investment and dedicated teams. If, on the other hand, it is not updated, we will lose the new functionalities of the tools and the programmers will eventually stop using the platform.

But while they may not be perfect for everyone, IDPs are not only facilitating better DevX, they are changing the way we think about software development and deployment. On the other hand, portals function as a centralized entry point that provides developers with self-service capabilities, a comprehensive software catalog and data-driven information to improve software quality and reliability. With all this, developers can focus on innovation and high-quality software development, while ensuring compliance with organizational policies and requirements.

In other words, they improve the development experience.