Transform a large-scale architecture - 3 - create the vision

Transform a large-scale architecture: Create the architecture vision

This blog post is part of the series "Transform a Large-Scale Architecture Guide," which includes the following parts:

• Before the Landing
• The First 30 Days, and the First Half Year
• Create the Vision
• Get the First Project Started
• Execution and Delivery
• Hiring, Promotion, and Performance Evaluation

The Vision

A vision is a mission statement that should be precise, easy to understand, and memorable. Defining a vision is a collective thinking exercise that requires a lot of understanding, time, and effort. There is no one way to define a vision, but the following are necessary:

Be Open-Minded and Help People Solve Their Pain Points

Be open-minded and always think about win-win solutions. Software is created by humans, and any changes to the current software product will involve people. Communication with people requires mindfulness about the conversations. The goal is to have honest conversations, but conversations can easily go nowhere.

Be a multiplier by asking questions to enable things. The current business may already have ten systems running for over ten years and believe changing them would be too disruptive. Redefining them may not bring necessary business value. Instead, think about why you are doing the same thing. Put that thinking aside and ask questions about the challenges of the existing systems and how you can help.

Understand the Current

Research the current systems to understand their relationships, functions, technologies, and business values. Use an Excel sheet to list all the functions and business use cases. Abstract them into different domains and establish the relationships among them.

Understanding the current also means having conversations with people familiar with the systems. Learn about how they are using them, what the patterns are, the current challenges, and any ideas for change. The conversations about the current will be fragmental but are representative of daily life.

Think from the current business perspective and connect them through business metrics data. If there isn't any, try to find something similar or representative.

Think Outside the Box

The current systems are still running, but maybe not very well. There are backgrounds on the existing business, and it's not just the technical reason why it is still here today. Think outside the technical box to understand the organizational, business, and industry growing history part of it.

The industry is evolving every day, and there are well-established technologies and newly innovated solutions or tools. For the domains identified from the current, think about industry trends, innovations, or new ideas. As an experienced engineer, pick the right trends for the current business to invest in, work with, or create new ones.

Keeping track of the latest trends in research areas and the industry can also help to keep the mind fresh and know where to find more detailed information when necessary. Joining open research channels, podcasts, conferences, etc. regularly can be helpful.

Value Proposition

Once you understand the current and think outside the box, there are apparently things the business can do better, either through using new technology or just creating a better product to achieve better results. But what are the better results, and how can you justify them through data?

Collecting data through the current is not easy since, if they have already understood the current metrics so easily, why are they not starting to change? The metrics data may be buried in various issues, notes, emails, or manual works, but not in a systematic way. Finding a representative data that is similar can also work. Maybe through a customer survey, data mining on current notes, issues.

Collecting data from the industry means reading reports and surveys on the industry-leading companies. Market researching companies like Gartner provide insight into who the leaders are in each domain and why they are the leaders. The leaders are usually innovators who educate the market on what are the things and provide insights on how can they use their product to achieve better results.

Use the industry and current data to propose the values, collect feedback, and reach agreement to fund the projects.

Architecture Framework

Architecture is an abstraction of each domain and the relationship among them. There are layers from bottom to top and connectors in between. Defining an architecture can be easy or hard depending on the level:

• Don't talk about too many details, create new concepts;
• The architecture boundary should be stable, not changed by details;
• The architecture should be flexible so that each domain can be prioritized differently.

Create New Concepts

There are always many details. Try to scope them into different domains and name them with different concepts. Verify the concepts through different conversations, document them, and communicate over-and-over to see people's reactions.

The concept name should not be completely new that nobody understands at first hearing. Try to find the concepts through people's talkings so that they can match their expectations more easily. Keep explaining new concepts over-and-over means education costs at latter phases.

The concepts should not leave out the technical details, including protocols, access and identity, databases, security considerations. Don't talk about them. Instead, focus on the concepts and what the use cases are.

Stable Architecture Boundary

Architecture is for guidance on technology. It is a high-level abstraction of the system. It should cover all the cases researched before and will cover the cases according to the industry research. Each domain architecture is an abstraction of the current business and technology and should not move in the many years to come.

The architecture is also forward-looking. Any new changes coming should be easily extendable to the current architecture, which means that the core concepts are unified, stable, and also extendable. Change to core concepts is a disaster to the whole system, so the concept and life-cycle around it need to be well-communicated and stabilized.

Each business domain is composed of the business cases around it. The core concepts need to be loosely-coupled. Each interface for each business domain should be stable and backward-compatible. It should have its business perspective, and the design should focus on its perspective. Even if there is a problem around each domain, it should be easily replaceable without affecting other domain services.

The book "Clean Architecture" has a diagram to illustrate the relationship.

Flexible Architecture

The business priority can and will change. The things planned at the beginning can and will change. Having a flexible architecture that adapts to change will affect execution efficiency.

The architecture relationship among each domain should be loosely-coupled and abstracted through interfaces. There are patterns like Strategy Pattern and Abstract Factory that can help create replaceable dependencies. When a new service is not ready or partly ready, having an abstraction will keep the architecture evolving.

Each domain should be flexible, and the lifecycle of each domain should also be replaceable. So when some part of the lifecycle is not actually there, create a simple one with little effort to help the business grow.

At the end of the day, the vision is about thought determination, that the business is doomed to change in an iterative manner. The above things are things to consider, but the key is about determination and bringing people along. Even if things are not there at the beginning, things can still happen.