IT is part of the business

During the design stage of a system, regardless of the field to which we refer, many factors are considered, which may depend on the intrinsic nature of the system. Among these factors we can name:

• User requirements: It is essential to understand the needs and expectations of the end users of the system. This involves clearly identifying and documenting functional requirements (what the system must do) and non-functional requirements (performance, security, usability, etc.). The user requirements provide a solid basis for the design of the system, what function it should fulfill and how it should do it.
• Scalability and ability to grow: It is important to consider the ability of the system to adapt and grow in the future. This includes assessing its ability to scale to handle larger volumes of data, users or transactions as demand increases. The design should be flexible and modular to allow for future expansions and upgrades without requiring drastic changes in structure or functionality.
• Performance and efficiency: Closely linked to the previous point; the system design must be efficient in terms of response time, resource utilization and processing capacity. This involves optimizing algorithms, minimizing system load, selecting appropriate technologies and performing performance tests to identify and address potential bottlenecks.
• System architecture: An appropriate system architecture must be defined. This involves making decisions about the overall structure of the system, the interaction between components, the distribution of tasks and the communication between modules. A well-thought-out architecture ensures the efficiency, scalability, reliability and maintainability of the system.
• Usability and user experience: The system design should focus on providing an intuitive and easy-to-use user experience. User interface design principles, accessibility, efficient navigation and clear presentation of information should be considered. Conducting usability testing and collecting user feedback during the design process helps to improve the overall user experience.
• Security and data protection: In many systems, data security and protection against external threats are critical issues. It is important to identify potential vulnerabilities and design appropriate security measures, such as authentication, authorization, data encryption and protection against cyber-attacks. The system design should consider sound security policies and practices to protect the integrity and confidentiality of information.
• Maintainability and extensibility: The system design should facilitate its maintenance over time. This involves using sound software development practices, following standards and conventions, documenting code, and providing a modular and well-organized structure. In addition, consideration should be given to the ability to extend and improve the system in the future without negatively affecting its stability.

And these are just a few of many other aspects to be considered.

The analysis of these factors and the correct selection of the solution for each of them will determine the success or failure of the system.

When implementing the solution or when evolving it, we may find dependencies and unwanted side effects, which may have been detected as a risk and have become a reality. You know, as Murphy’s law says “if something can go wrong, it will go wrong”. In other words, it suggests that if there is a chance of something going wrong in each situation, it is very likely to happen. But perhaps we are making this likelihood greater by not giving enough importance to certain issues. Maybe it’s not Murphy’s fault.

The above list is not exhaustive, there are several factors that have not been named and one of them is the existing communication in the environment, and it is often key to the successful implementation and evolution of systems.

The designed system coexists and will communicate with other systems, and there may be undesired effects if this aspect is not considered as we saw some time ago when we talk about Systems Thinking; but there are also internal communications between teams and people that must be considered.

Introducing Conway

In 1968, Melvin Conway stated that “organizations that design systems … are inevitably constrained by the communication structure of their own teams” and this is known as Conway’s Law. In other words, the architecture of a system reflects the structure of the organization that builds it. And if the reader wonders whether this law is still valid after more than five decades, I encourage you to analyze how your company’s systems work to realize that it is as valid as the first day.

This law has profound implications for the design and development of systems, especially technology systems, including software and data architecture. It is important to note that we are talking about the communication structure, not the organizational structure, and it is not always the same thing. The fact that two departments of the same company are located in the same space will cause them to communicate with each other, even if they are not related in the organization chart or in the value chain flows, and this in turn will impact the system designs.

When software development teams are organized functionally, i.e., with separate specialists in different areas, the resulting system will most likely be fragmented into modules that reflect the communication between those teams. For example, if one development team is responsible for business logic, another for user interface design, and another for the data persistence layer, it is likely that the system architecture will clearly separate these aspects. But in this case, it is also likely that the data architecture is fragmented into different information silos. Each team may have its own database or content repository without taking into account the other actors involved, which may hinder interoperability and information sharing between the different components of the system. This can lead to problems of data redundancy, lack of integrity and difficulties in obtaining a complete and accurate view of the information.

On the other hand, if teams are organized in a more integrated and collaborative manner, the resulting system is more likely to have a more cohesive and flexible architecture. Cross-functional teams, where members from different disciplines work together, tend to generate systems with better integration between the different components. This is because fluid communication between teams fosters shared understanding of the overall architecture and promotes collaboration to design a coherent structure. This cross-functionality we see for example in scrum teams.

Fluid communication between team members will enable the identification of patterns and reusable components, resulting in a more coherent and modular architecture. For example, design patterns can be established that promote separation of concerns and facilitate system scalability and maintainability.

As an example of Conway’s Law we could take the development of a web application. If a development team is divided into specialized front-end and back-end sub-teams, the application architecture will most likely reflect this division. The presentation layer will be more focused on the user interface and user experience, while the server layer will focus on data processing and business logic without regard for each other. This can result in a lack of cohesion and difficulties in making changes or improvements to the system due to existing dependencies. In contrast, if the team works more collaboratively, with members having both front-end and back-end expertise, the application architecture is more likely to be more modular, scalable and easier to maintain.

In addition, when development teams work collaboratively and data is considered as a shared resource (the data, not the database), the data architecture is more likely to be consistent. Teams can, for example, adopt a service-oriented approach, where different system components interact through well-defined interfaces and share appropriate mechanisms for data synchronization and consistency.

In a microservices system, each service is responsible for a specific part of the functionality and has its own database. This reflects the organizational structure in which independent teams are responsible for individual services. While this can offer advantages in terms of scalability and team autonomy, it also introduces challenges in managing data consistency and information integrity.

To overcome these challenges, it is necessary to establish consistent data design practices and use appropriate synchronization and communication mechanisms between services. This involves establishing clear contracts and defining communication interfaces that allow services to exchange data reliably and consistently. In addition, techniques such as data replication and the implementation of consistency patterns can be implemented to ensure that information is kept up-to-date and consistent throughout the system.

It is important to note that Conway’s Law does not imply that the organization’s communication structure is immutable. In fact, it is beneficial to adapt the organizational structure (including geographically) to facilitate the creation of a more effective system architecture. This implies that development teams and organizational structures must be constantly evolving and adjusting to align their objectives with system requirements. This of course also affects the data design which must go hand in hand with the system architecture and adjustments may need to be made to the data architecture to reflect the new organizational structure and enable effective collaboration.

Conway’s Law also has implications for communication and collaboration between geographically distributed teams. In these cases, it is critical to establish effective communication channels and promote a culture of remote collaboration. If teams are not aligned and do not communicate efficiently, the system architecture is likely to be negatively affected. In contrast, when agile practices are established and appropriate collaboration tools are used, distributed teams can overcome challenges and achieve a consistent, high-quality architecture.

Dancing with Conway

We have already seen how Conway’s law states that the architecture of a system reflects the communication structure of the teams that build it; but how to deal with it? Completely undoing this law can be difficult, since it implies changing the organizational structure and the existing communication dynamics in an organization, which impacts the company culture. However, it is possible to perform certain maneuvers or actions that can mitigate the negative effects of the law or allow a more effective adaptation to the needs of the system. Some strategies that can be used could be:

• Reorganization of teams: A key point is the organization, function and structure of teams. The idea is to group team members according to the desired architecture of the system, rather than on the basis of individual roles, and to establish precise communication channels between them. In this way, collaboration between members from different disciplines is encouraged and a more coherent architecture is promoted. For example, instead of having separate front-end and back-end teams, multidisciplinary teams can be formed that include developers, designers and database specialists. If we want two modules to be completely decoupled, it is best to decouple the teams working on it, even physically/geographically if possible. If we want two modules not to share the same database, it is best not to share the same person with database skills between the two modules.

• Fluid communication and collaboration: Promoting fluid communication and collaboration among team members is essential to undo Conway’s Law. This involves establishing effective communication channels, holding regular meetings and fostering a culture of collaboration. The idea is for teams to work together in an integrated way, sharing knowledge and making joint decisions on system architecture. This will enable a shared understanding and greater consistency in system design and development.
• Focus on architecture: It is critical to have a clear vision of the desired system architecture from the outset and to communicate it to all team members. This includes defining architectural principles, establishing design patterns and documenting key decisions. By focusing on the architecture and establishing clear direction, you can mitigate the negative effects of Conway’s Law and ensure that the system structure reflects business needs and technical requirements.
• Agile and DevOps practices: Adopting agile and DevOps practices can help undo Conway’s Law by fostering collaboration, continuous communication, and rapid and frequent software delivery. These methodologies promote multidisciplinary team building, process automation and continuous improvement. By working in short cycles and allowing constant feedback, adjustments to the architecture and organizational structure can be made more quickly and efficiently.
• Use of services: Adopting an architecture based on independent services can help mitigate the effects of Conway’s Law by enabling greater modularity and reuse of components. By clearly defining interfaces and separating responsibilities into independent services, the dependency of the organizational structure on the system architecture can be reduced. This allows different teams to work in parallel and autonomously (and sometimes it has to be forced to be truly autonomous and independent), collaborating through well-defined interfaces. With this approach, the integration of the different system components is facilitated and the dependency of the organizational structure on the software architecture is reduced.

It is important to note that completely undoing Conway’s Law can be challenging, especially in large, established organizations. However, by implementing these strategies, it is possible to mitigate the negative effects and enable a more effective adaptation of the system architecture to the business needs.

Similarly, it should be kept in mind that undoing Conway’s Law requires a cultural and organizational change across the enterprise. It requires a commitment and willingness to adapt to the way teams communicate, collaborate and organize. In addition, there must be an ongoing focus on continuous improvement and alignment of the organizational structure with the goals and requirements of the technology system.

So remember, it may not be Murphy, it may be Conway.