The Zen of Architects1 simply states that for the beginner architect, there are many options of doing pretty much anything. For the master architect, however, there are only a few good options, and typically only one. 1. https://en.wikipedia.org/wiki/Zen_Mind,_Beginner’s_Mind (Location 501) #✂️

The Method is a simple and effective analysis and design technique. You can express The Method as a formula: The Method = System Design + Project Design With system design, The Method lays out a way of breaking down a big system into small modular components. The Method offers guidelines for the structure, role, and semantics of the components and how these components should interact. The result is the architecture of the system. With project design, The Method helps you provide management with several options for building the system. Each option is some combination of schedule, cost, and risk. Each option also serves as the system assembly instructions, and it sets up the project for execution and tracking. (Location 529) #✂️

Project design is the second part of the book and is far more important for success than system design. (Location 535) #✂️

Design validation is critical because an organization should not risk having a team start developing against an inadequate architecture or developing a system the organization cannot afford to build. (Location 542) #✂️

Early validation of the design is imperative. (Location 551) #✂️

Ideally, one week into the project, you must know if the architecture is going to hold water (or not). (Location 552) #✂️

Note that I am referring here to the system design, the architecture, not the detailed design of the system. Detailed design produces for each component in the architecture the key implementation artifacts, such as interfaces, class hierarchies, and data contracts. Detailed design takes longer to produce, can be done during the project execution, and may change as the system is constructed or evolved. (Location 554) #✂️

Similarly, you must validate your project design. Running out of time or running over budget (or both) mid-project is simply unacceptable. (Location 558) #✂️

No project should fail because it did not have enough time or resources from the start. This book shows you how to accurately calculate the project duration and costs and how to drive educated decisions. (Location 563) #✂️

Using The Method, you can produce an entire system design in mere days, typically in three to five days, with project design taking similar time. (Location 565) #✂️

In general, design is not time-consuming (as opposed to implementation). Building architects charge hourly and often work only a week or two at most designing a house. Constructing a house from the architect’s design might take an agonizing two to three years of working with contractors, and yet the architect did not take long to produce the architecture. (Location 572) #✂️

The time crunch also helps avoid design gold plating. Parkinson’s law2 states that work always expands to fill the allotted time. Given 10 days to complete a design that could be completed in five days, the architect will likely work on the design for 10 days. (Location 575) #✂️

2. Cyril N. Parkinson, “Parkinson’s Law,” The Economist (November 19, 1955). (Location 579) #✂️

Analysis-paralysis is a predicament that occurs when someone (or a group) who is otherwise capable, clever, and even hardworking (as are most software architects) is stuck in a seemingly endless cycle of analysis, design, new revelations, and back to more analysis. (Location 582) #✂️

The main reason for the paralysis is being unaware of the design decision tree for both the system and the project. The design decision tree is a general concept that applies to all design tasks, not just in software engineering. The design of any complex entity is a collection of many smaller design decisions, arranged hierarchically in a tree-like structure. (Location 585) #✂️

Invariably, at some point, a downstream design decision will invalidate a prior decision; all decisions made in between these two points will be invalid. Designing this way is akin to performing a bubble sort of the design decision tree. Since bubble sort roughly involves as many operations as the square of the number of elements involved, the penalty is severe. (Location 591) #✂️

Only after you have designed the system is there any point in designing the project to build that system. (Location 602) #✂️

One of the most valuable techniques in pruning the decision tree is the application of constraints. As pointed by Dr. Fredrick Brooks,3 contrary to common wisdom or intuition, the worst design problem is a clean canvas. (Location 605) #✂️

3. Frederick P. Brooks Jr., The Design of Design: Essays from a Computer Scientist (Upper Saddle River, NJ: Addison-Wesley, 2010). (Location 612) #✂️

Software architecture is the high-level design and structure of the software system. (Location 654) #✂️

The act of identifying the constituent components of a system is called system decomposition. (Location 658) #✂️

In a modern system and in this book, services (as in service-orientation) are the most granular unit of the architecture. However, the technology used to implement the components and their details (such as interfaces, operations, and class hierarchies) are detailed design aspects, not system decomposition. (Location 662) #✂️

Functional decomposition decomposes a system into its building blocks based on the functionality of the system. (Location 670) #✂️

At the very least, functional decomposition couples services to the requirements because the services are a reflection of the requirements. Any change in the required functionality imposes a change on the functional services. (Location 675) #✂️

Because functional decomposition is also decomposition based on time (call A and then call B), it effectively precludes individual reuse of services. Suppose another system also needs a B service (such as Billing). Built into the fabric of B is the notion that it was called after an A and before a C service (such as first Invoicing, and only then Billing against an invoice, and finally Shipping). (Location 684) #✂️

One way of performing functional decomposition is to have as many services as there are variations of the functionalities. This decomposition leads to an explosion of services, since a decently sized system may have hundreds of functionalities. Not only do you have too many services, but these services often duplicate a lot of the common functionality, each customized to their case. (Location 702) #✂️

Another functional decomposition approach is to lump all possible ways of performing the operations into mega services. This leads to bloating in the size of the services, making them overly complex and impossible to maintain. (Location 706) #✂️

Functional decomposition, therefore, tends to make services either too big and too few or too small and too many. You often see both afflictions side by side in the same system. (Location 709) #✂️

Functional decomposition often leads to flattening of the system hierarchy. Since each service or building block is devoted to a specific functionality, someone must combine these discrete functionalities into a required behavior. That someone is often the client. (Location 714) #✂️

The hallmark of a bad design is when any change to the system affects the client. Ideally, the client and services should be able to evolve independently. (Location 732) #✂️

Yet, when designed as in Figure 2-1, you are forced to pollute the client with the business logic of sequencing, ordering, error compensation, and duration of the calls. (Location 734) #✂️

Cyclomatic complexity measures the number of independent paths through the code of a class or service. The more the internals are convoluted and coupled, the higher the cyclomatic complexity score. (Location 747) #✂️

Another problem with the decomposition of Figure 2-1 is that it requires multiple points of entry to the system. The client (or clients) needs to enter the system in three places: once for the A, then for the B, then for the C service. This means there are multiple places to worry about authentication, authorization, scalability, instance management, transaction propagation, identities, hosting, and so on. (Location 761) #✂️

The advantage of doing so is that you get to keep the clients simple and even asynchronous: the clients issue the call to the A service. The A service then calls B, and B calls C. The problem now is that the functional services are coupled to each other and to the order of the functional calls. For example, you can call the Billing service only after the Invoicing service but before the Shipping service. In the case of Figure 2-3, built into the A service is the knowledge that it needs to call the B service. The B service can be called only after the A service and before the C service. A change in the required ordering of the calls is likely to affect all services up and down the chain because their implementation will have to change to reflect the new required order. (Location 773) #✂️

Chaining functionality leads to bloated services. (Location 801) #✂️

Functional decomposition holds an almost irresistible allure. It looks like a simple and clear way of designing the system, requiring you to simply list the required functionalities and then create a component in your architecture for each. Functional decomposition (and its kin, the domain decomposition discussed later) is how most systems are designed. (Location 842) #✂️

At all costs, you must resist the temptations of functional decomposition. (Location 847) #✂️

Consider building a house functionally, as if it were a software system. You start by listing all the required functionalities of the house, such as cooking, playing, resting, sleeping, and so on. You then create an actual component in the architecture for each functionality, (Location 875) #✂️

The derision in these pages does not mean functional decomposition is a bad idea. Functional decomposition has a place—it is a decent requirements discovery technique. (Location 893) #✂️

domain decomposition: decomposing a system into building blocks based on the business domains, such as sales, engineering, accounting, and shipping. Sadly, domain decomposition such as Figure 2-7 shows is even worse than the functional decomposition of Figure 2-6 (Location 901) #✂️

The problem is that you can never deploy a single feature in isolation. There is no business value in Billing independent from Invoicing and Shipping. (Location 944) #✂️

A crucial flaw of both functional and domain decomposition has to do with testing. With such designs, the level of coupling and complexity is so high that the only kind of testing developers can do is unit testing. However, that does not make unit testing important, and it is merely another example of the streetlight effect1 (i.e., searching for something where it is easiest to look). 1. https://en.wikipedia.org/wiki/Streetlight_effect (Location 968) #✂️

Contrary to intuition, software requires design even more than physical systems do. The reason is simple: complexity. The complexity of physical systems such as typical houses is capped by physical constraints. (Location 996) #✂️

Without such natural physical restraints, complexity in software systems can get quickly out of control. The only way to rein in that complexity is to apply good engineering methods, of which design and process are paramount. (Location 1001) #✂️

Decompose based on volatility. Volatility-based decomposition identifies areas of potential change and encapsulates those into services or system building blocks. You then implement the required behavior as the interaction between the encapsulated areas of volatility. (Location 1091) #✂️

Functional decomposition therefore tends to maximize the effect of the change. Since most software systems are designed functionally, change is often painful and expensive, and the system is likely to resonate with the change. (Location 1103) #✂️

Accommodating change is the real reason you must avoid functional decomposition. (Location 1105) #✂️

A functional decomposition of your own body would have components for every task you are required to do, from driving to programming to presenting, yet your body does not have any such components. (Location 1139) #✂️

Volatility-based decomposition lends well to regression testing. The reduction in the number of components, the reduction in the size of components, and the simplification of the interactions between components all drastically reduce the complexity of the system. This makes it feasible to write regression testing that tests the system end to end, tests each subsystem individually, and eventually tests independent components. (Location 1157) #✂️

In 1972, David Parnas (an early pioneer of software engineering) published a seminal paper called “On the Criteria to Be Used in Decomposing Systems into Modules.”a This short, five-page paper contains most elements of modern software engineering, including encapsulation, information hiding, cohesion, modules, and loose coupling. (Location 1165) #✂️

It is the 2% problem again: it is not worth your while learning how to fix that sink if it is clogged less than 2% of the time. The moral is that when you spend 2% of your time on any complex task, you will never be any good at it. (Location 1199) #✂️

In 1999, David Dunning and Justin Kruger published their research2 demonstrating conclusively that people unskilled in a domain tend to look down on it, thinking it is less complex, risky, or demanding than it truly is. This cognitive bias has nothing to do with intelligence or expertise in other domains. If you are unskilled in something, you never assume it is more complex than it is, you assume it is less! 2. Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology 77, no. 6 (1999): 1121–1134. (Location 1215) #✂️

Not everything that is variable is also volatile. (Location 1237) #✂️

You resort to encapsulating a volatility at the system design level only when it is open-ended and, unless encapsulated in a component of the architecture, would be very expensive to contain. (Location 1237) #✂️

Variability, on the other hand, describes those aspects that you can easily handle in your code using conditional logic. (Location 1238) #✂️

When searching for volatility, you should be on the lookout for the kind of changes or risks that would have ripple effects across the system. (Location 1239) #✂️

There is a simple technique I call axes of volatility. This technique examines the ways the system is used by customers. (Location 1243) #✂️

Customer in this context refers to a consumer of the system, which could be a single user or a whole other business entity. (Location 1244) #✂️

In any business, there are only two ways your system could face change: the first axis is at the same customer over time. Even if presently the system is perfectly aligned with a particular customer’s needs, over time, that customer’s business context will change. (Location 1245) #✂️

The tendency of a solution to change the requirements against which it was developed was first observed by the 19th-century English economist William Jevons with regard to coal production, and it is referred to since as the Jevons paradox. Other manifestations are the increase in paper consumption with the digital office and the worsening traffic congestion following an increase in road capacity. (Location 1250) #✂️

The second way change could come is at the same time across customers. (Location 1253) #✂️

When searching for potential volatility in interviews, you will find it very helpful to phrase the questions in terms of the axes of volatility (same customer over time, all customers at the same point in time). (Location 1256) #✂️

Almost always, the axes should be independent. Something that changes for one customer over time should not change as much across all customers at the same point in time, and vice versa. If areas of change cannot be isolated to one of the axes, it often indicates a functional decomposition in disguise. (Location 1270) #✂️

Since most requirements specifications are chock-full of solutions masquerading as requirements, functional decomposition absolutely maximizes your pain. You will forever be chasing the ever-evolving solutions, never recognizing the true underlying requirements. (Location 1325) #✂️

The fact that requirements specifications have all those solutions masquerading as requirements is actually a blessing in disguise because you can generalize the example of cooking in the house into a bona fide analysis technique for discovering areas of volatility. (Location 1327) #✂️

Prior to decomposing a system and creating an architecture, you should simply compile a list of the candidate areas of volatility as a natural part of requirements gathering and analysis. (Location 1333) #✂️

During system decomposition, you must identify both the areas of volatility to encapsulate and those not to encapsulate (e.g., the nature of the business). Sometimes, you will have initial difficulty in telling these apart. There are two simple indicators if something that could change is indeed part of the nature of the business. The first indicator is that the possible change is rare. Yes, it could happen, but the likelihood of it happening is very low. The second indicator is that any attempt to encapsulate the change can only be done poorly. No practical amount of investment in time or effort will properly encapsulate the aspect in a way of which you can be proud. (Location 1470) #✂️

Another useful technique for identifying volatilities is to try to design a system for your competitor (or another division in your company). (Location 1508) #✂️

Ask yourself the following question: Can Federal Express use the software system UPS is using? Can UPS use the system Federal Express wants to build? If the likely answer is no, start listing all the barriers for such a reuse or extensibility. While both companies perform in the abstract the same service, the way they conduct their business is different. (Location 1511) #✂️

The opposite case is also true. If you and your competitor (and even better, all competitors) do some activity or sequence the same way, and there is no chance of your system doing it any other way, then there is no need to allocate a component in the architecture for that activity. To do so would create a functional decomposition. When you encounter something your competitors do identically, more likely than not, it represents the nature of the business, and as discussed previously, you should not encapsulate it. (Location 1518) #✂️

Volatility is intimately related to longevity. The longer the company or the application has been doing something the same way, the higher the likelihood the company will keep doing it the same way. Put differently, the longer things do not change, the longer they have until they do change or are replaced. You must put forward a design that accommodates such changes, even if at first glance such changes are independent of the current requirements. (Location 1523) #✂️

You can even guesstimate how long it will be until such a change is likely to take place using a simple heuristic: the ability of the organization (or the customer or the market) to instigate or absorb a change is more or less constant because it is tied to the nature of the business. (Location 1526) #✂️

Before diving into architecture, consider requirements. Most projects, if they even bother to capture the requirements, use functional requirements. Functional requirements simply state the required functionality, such as “The system should do A.” This is actually a poor way of specifying requirements, because it leaves the system’s implementation of the A functionality open for interpretation. (Location 1578) #✂️

Requirements should capture the required behavior rather than the required functionality. You should specify how the system is required to operate as opposed to what it should do, which is arguably the essence of requirements gathering. (Location 1585) #✂️

A use case is an expression of required behavior—that is, how the system is required to go about accomplishing some work and adding value to the business. As such, a use case is a particular sequence of activities in the system. Use cases tend to be verbose and descriptive. They can describe end-user interactions with the system, or the system’s interactions with other systems, or back-end processing. (Location 1589) #✂️

The best way of capturing a use case is graphically, with a diagram (Figure 3-1). Humans perform image processing astonishingly quickly, because almost half the human brain is a massive video processing unit. (Location 1599) #✂️

My rule of thumb: The presence of a nested “if” tells you that you should draw the use case. (Location 1605) #✂️

The Method prefers activity diagrams1 for graphical representation of use cases, primarily because activity diagrams can capture time-critical aspects of behavior, something that flowcharts and other diagrams are incapable of doing. (Location 1611) #✂️

Software systems are typically designed in layers, and The Method relies heavily on layers. Layers allow you to layer encapsulation. Each layer encapsulates its own volatilities from the layers above and the volatilities in the layers below. (Location 1626) #✂️

Even simple systems should be designed in layers to gain the benefit of encapsulation. In theory, the more layers, the better the encapsulation. Practical systems will have only a handful of layers, terminating with a layer of actual physical resources such as a data storage or a message queue. (Location 1631) #✂️

The preferred way of crossing layers is by calling services. While you certainly can benefit from the structure of The Method and volatility-based decomposition even with regular classes, relying on services provides distinct advantages. Which technology and platform you use to implement your services is a secondary concern. (Location 1634) #✂️

When you do use services (as long as the technology you chose allows), you immediately gain the following benefits: Scalability. (Location 1636) #✂️

Security. All service-oriented platforms treat security as a first-class aspect. (Location 1640) #✂️

Throughput and availability. Services can accept calls over queues, allowing you to handle a very large volume of messages by simply queuing up the excess load. (Location 1642) #✂️

Responsiveness. Services can throttle the calls into a buffer to avoid maxing out the system. (Location 1644) #✂️

Reliability. Clients and services can use some reliable messaging protocol to guarantee delivery, handle network connectivity issues, and even order the calls. (Location 1645) #✂️

Consistency. The services can all participate in the same unit of work, either in a transaction (when supported by the infrastructure) or in a coordinated business transaction that is eventually consistent. (Location 1646) #✂️

Synchronization. The calls to the service can be automatically synchronized even if the clients use multiple concurrent threads. (Location 1649) #✂️

The Method calls for four layers in the system architecture. These layers conform to some classic software engineering practices. (Location 1651) #✂️

The top layer in architecture is the client layer, also known as the presentation layer. I find the term “presentation” to be somewhat misleading. “Presentation” implies some information is being presented to human users, as if that is all that is expected from the top layer. The elements in the client layer may very well be end-user applications, but they can also be other systems interacting with your system. (Location 1656) #✂️

The client layer also encapsulates the potential volatility in Clients. Your system now and in the future across the axes of volatility may have different Clients such as desktop applications, web portals, mobile apps, holograms and augmented reality, APIs, administration applications, and so on. (Location 1667) #✂️

The business logic layer encapsulates the volatility in the system’s business logic. This layer implements the system’s required behavior, which, as mentioned previously, is best expressed in use cases. (Location 1673) #✂️

Use cases, however, are volatile, across both customers and time. Since a use case contains a sequence of activities in the system, a particular use case can change in only two ways: Either the sequence itself changes or the activities within the use case change. (Location 1675) #✂️

Both the sequence and the activities are volatile, and in The Method these volatilities are encapsulated in specific components called Managers and Engines. Manager components encapsulate the volatility in the sequence, whereas Engine components encapsulate the volatility in the activity. (Location 1687) #✂️

Since use cases are often related, Managers tend to encapsulate a family of logically related use cases, such as those in a particular subsystem. (Location 1693) #✂️

Since you can have great volatility in the sequence without any volatility in the activities of the sequence (see Figure 3-5), Managers may use zero or more Engines. Engines may be shared between Managers because you could perform an activity in one use case on behalf of one Manager and then perform the same activity for another Manager in a separate use case. (Location 1698) #✂️

You should design Engines with reuse in mind. However, if two Managers use two different Engines to perform the same activity, you either have functional decomposition on your hands or you have missed some activity volatility. (Location 1701) #✂️

The aptly named resource access layer encapsulates the volatility in accessing a resource, and the components in this layer are called ResourceAccess. (Location 1704) #✂️

While the motivation behind the resource access layer is readily evident and many systems incorporate some form of an access layer, most such layers end up exposing the underlying volatility by creating a ResourceAccess contract that resembles I/O operations or that is CRUD-like. For example, if your ResourceAccess service contract contains operations such as Select(), Insert(), and Delete(), the underlying resource is most likely a database. If you later change the database to a distributed cloud-based hash table, that database-access-like contract will become useless, and a new contract is required. Changing the contract affects every Engine and Manager that has used the ResourceAccess component. (Location 1710) #✂️

A well-designed ResourceAccess component exposes in its contract the atomic business verbs around a resource. (Location 1721) #✂️

The Method refers to these indivisible activities as atomic business verbs. (Location 1725) #✂️

Atomic business verbs are practically immutable because they relate strongly to the nature of the business, which, as discussed in Chapter 2, hardly ever changes. (Location 1728) #✂️

ResourceAccess services can be shared between Managers and Engines. You should explicitly design ResourceAccess components with this reuse in mind. If two Managers or two Engines cannot use the same ResourceAccess service when accessing the same resource or have some need for specific access, perhaps you did not encapsulate some access volatility or did not isolate the atomic business verbs correctly. (Location 1734) #✂️

The resource layer contains the actual physical Resources on which the system relies, such as a database, file system, a cache, or a message queue. In The Method, the Resource can be internal to the system or outside the system. Often, the Resource is a whole system in its own right, but to your system it appears as just a Resource. (Location 1738) #✂️

The Method recommends the following conventions for naming them: Names of services must be two-part compound words written in Pascal case. The suffix of the name is always the service’s type—for example, Manager, Engine, or Access (for ResourceAccess). The prefix varies with the type of service. –  For Managers, the prefix should be a noun associated with the encapsulated volatility in the use cases. –  For Engines, the prefix should be a noun describing the encapsulated activity. –  For ResourceAccess, the prefix should be a noun associated with the Resource, such as data that the service provides to the consuming use cases. Gerunds (a gerund is a noun created by tacking “ing” onto a verb) should be used as a prefix only in with Engines. The use of gerunds elsewhere in the business or access layers usually signals functional decomposition. Atomic business verbs should not be used in a prefix for a service name. These verbs should be confined to operation names in contracts interfacing with the resource access layer. (Location 1757) #✂️

The layers of services and resources in the architecture loosely correspond to the four English questions of “who,” “what,” “how,” and “where.” “Who” interacts with the system is in the Clients, “what” is required of the system is in Managers, “how” the system performs business activities is in Engines, “how” the system accesses Resources is in ResourceAccess, and “where” the system state is in Resources (see Figure 3-7). (Location 1778) #✂️

If your design contains a large number of Engines, you may have inadvertently done a functional decomposition. (Location 1803) #✂️

If your system has eight Managers, then you have already failed to produce a good design: The large number of Managers strongly indicates you have done a functional or domain decomposition. (Location 1809) #✂️

In a well-designed system, volatility should decrease top-down across the layers. Clients are very volatile. (Location 1821) #✂️

Managers do change, but not as much as their Clients. Managers change when the use cases—the required behavior of the system—change. (Location 1824) #✂️

Engines are less volatile than Managers. For an Engine to change, your business must change the way it is performing some activity, which is more uncommon than changing the sequencing of activities. (Location 1825) #✂️

ResourceAccess services are even less volatile than Engines. How often do you change the way you access a Resource or, for that matter, change the Resource? (Location 1826) #✂️

Resources are the least volatile components, changing at a glacial pace compared with the rest of the system. (Location 1829) #✂️

Reuse, unlike volatility, should increase going down the layers. (Location 1833) #✂️

Clients are hardly ever reusable. A Client application is typically developed for a particular type of platform and market and cannot be reused. (Location 1833) #✂️

Managers are reusable because you can use the same Manager and use cases from multiple Clients. (Location 1835) #✂️

Engines are even more reusable than Managers because the same Engine could be called by multiple Managers, in different use cases, to perform the same activity. (Location 1836) #✂️

ResourceAccess components are very reusable because they can be called by Engines and Managers. (Location 1838) #✂️

The Resources are the most reusable element in any well-designed systems. (Location 1838) #✂️

Managers can fall into one of three categories: expensive, expendable, and almost expendable. You can distinguish the category to which a Manager belongs by the way you respond when you are asked to change it. (Location 1841) #✂️

If your response is to fight the change, to fear its cost, to argue against the change, and so forth, then the Manager was clearly expensive and not expendable. An expensive Manager indicates that the Manager is too big, likely due to functional decomposition. If your response to the change request is just to shrug it off, thinking little of it, the Manager is pass-through and expendable. (Location 1843) #✂️

Expendable Managers are always a design flaw and a distortion of the architecture. They often exist only to satisfy the design guidelines without any real need for encapsulating use case volatility. (Location 1846) #✂️

A well-designed Manager service should be almost expendable. (Location 1851) #✂️

The Managers, Engines, and ResourceAccess are all services on their own right. A cohesive interaction between the Manager, Engines, and ResourceAccess may constitute a single logical service to external consumers. You can view such a set of interacting services as a logical subsystem. You group these together as a vertical slice of your system (Figure 3-8), where each vertical slice implements a corresponding set of use cases. (Location 1853) #✂️

Avoid over-partitioning your system into subsystems. Most systems should have only a handful of subsystems. Likewise, you should limit the number of Managers per subsystem to three. (Location 1858) #✂️

Design iteratively, build incrementally. (Location 1870) #✂️

There are two reasons why you can build only incrementally, and not iteratively. First, building iteratively is horrendously wasteful and difficult (turning a motorcycle into a car is much more difficult than just building a car). Second, and much more importantly, the intermediate iterations do not have any business value. If the customer wants a car to take the kids to school, what would the customer do with a motorcycle and why should the customer pay for it? (Location 1885) #✂️

The vertical slices of the system also enable you to accommodate extensibility. The correct way of extending any system is not by opening it up and hammering on existing components. (Location 1898) #✂️

In common usage, microservices correspond to domains or subsystems—that is, to the slices (red boxes) of Figure 3-8. There are three problems with this idea as practiced today. (Location 1923) #✂️

The first problem is the implied constraint on the number of services. If smaller services are better than larger services, why stop at the subsystem level? (Location 1925) #✂️

The second problem is the widespread use of functional decomposition in microservice design by the industry at large. (Location 1932) #✂️

The third problem relates to communication protocols. Although the choice of communication protocols has more to do with detailed design than with architecture, the effect of the choice is worth a passing comment here. (Location 1937) #✂️

As a general principle, in any well-designed system you should never use the same communication mechanism both internally and externally. (Location 1941) #✂️

Any layered architecture can have one of two possible operational models: open or closed. (Location 1959) #✂️

In an open architecture, any component can call any other component regardless of the layer in which the components reside. (Location 1962) #✂️

Calling sideways in this way is almost always the result of functional decomposition at the Managers level. (Location 1981) #✂️

When using open architecture, there is hardly any benefit of having architectural layers in the first place. In general, in software engineering, trading encapsulation for flexibility is a bad trade. (Location 1986) #✂️

In a closed architecture, you strive to maximize the benefits of the layers by disallowing calling up between layers and sideways within layers. (Location 1988) #✂️

Closed architecture promotes decoupling by trading flexibility for encapsulation. In general, that is a better trade than the other way around. (Location 1992) #✂️

A semi-closed/semi-open architecture allows calling more than one layer down. This, again, is a trade of encapsulation for flexibility and performance and, in general, is a trade to avoid. (Location 1997) #✂️

Notably, the use of semi-closed/semi-open architecture is justified in two classic cases. This first case occurs when you design some key piece of infrastructure, and you must squeeze every ounce of performance from it. (Location 1999) #✂️

The second case occurs within a codebase that hardly ever changes. (Location 2004) #✂️

In a closed architecture, Utilities pose a challenge. Consider Logging, a service used for recording run-time events. If you classify Logging as a Resource, then the ResourceAccess can use it, but the Managers cannot. If you place Logging at the same level as the Managers, only the Clients can log. The same goes for Security or Diagnostics—services that almost all other components require. In short, there is no good location for Utilities among the layers of a closed architecture. The Method places Utilities in a vertical bar on the side of the layers (see Figure 3-4). This bar cuts across all layers, allowing any component in the architecture to use any Utility. (Location 2016) #✂️

To qualify as a Utility, the component must pass a simple litmus test: Can the component plausibly be used in any other system, such as a smart cappuccino machine? (Location 2026) #✂️

This next guideline may be implied, but it is important enough to state explicitly. Because they are in the same layer, both Managers and Engines can call ResourceAccess services without violating the closed architecture (see Figure 3-4). Allowing Managers to call ResourceAccess is also implied from the section defining Managers and Engines. A Manager that uses no Engines must be able to access the underlying Resources. (Location 2033) #✂️

Managers can directly call Engines. The separation between Managers and Engines is almost at the detailed design level. Engines are really just an expression of the Strategy design pattern6 used to implement the activities within the Managers’ workflows. Therefore, Manager-to-Engine calls are not truly sideways calls, as is the case with Manager-to-Manager calls. (Location 2038) #✂️

While Managers should not call directly sideways to other Managers, a Manager can queue a call to another Manager. (Location 2046) #✂️

Using The Method structure, when a Manager queues a call to another Manager, the proxy is a ResourceAccess to the underlying Resource, the queue; that is, the call actually goes down, not sideways. (Location 2051) #✂️

The semantic explanation involves the nature of use cases. Business systems quite commonly have one use case that triggers a latent, much-deferred execution of another use case. For example, imagine a system in which a Manager executing a use case must save some system state for analysis at the end of the month. Without interrupting its flow, the Manager could queue the analysis request to another Manager. The second Manager could dequeue at the month’s end and perform its analysis workflow. The two use cases are independent and decoupled on the timeline. (Location 2054) #✂️

If you do one of the things on this list, you will likely live to regret it. Treat any violation of these rules as a red flag and investigate further to see what you are missing: (Location 2078) #✂️

Another universal design rule is that all good architectures are symmetric. (Location 2119) #✂️

The quest for symmetry, however, is only at the architecture level, not in detailed design. (Location 2121) #✂️

The symmetry in software systems manifests in repeated call patterns across use cases. You should expect symmetry, and its absence is a cause for concern. (Location 2124) #✂️

For example, suppose a Manager implements four use cases, three of which publish an event with the Pub/Sub service and the fourth of which does not. That break of symmetry is a design smell. Why is the fourth case different? What are you missing or overdoing? Is that Manager a real Manager, or is it a functionally decomposed component without volatility? Symmetry can also be broken by the presence of something, not just by its absence. (Location 2125) #✂️

But how do you know the composition of these components at run-time adequately satisfies all the requirements? (Location 2135) #✂️

Design validation and composition, as you will see, are intimately related. (Location 2137) #✂️

Never design against the requirements. (Location 2157) #✂️

As discussed in Chapter 3, the correct way of capturing the requirements is in the form of use cases: the required set of behaviors of the system. (Location 2166) #✂️

The core use cases represent the essence of the business of the system. As discussed in Chapter 2, the nature of the business hardly ever changes, and the same goes for the core use cases. (Location 2187) #✂️

Most systems have as few as two or three core use cases, and the number seldom exceeds six. (Location 2193) #✂️

A core use case will almost always be some kind of an abstraction of other use cases, and it may even require a new term or name to differentiate it from the rest. (Location 2199) #✂️

The whole point of requirements analysis is to recognize the core use cases (and the areas of volatility). (Location 2202) #✂️

Your mission as an architect is to identify the smallest set of components that you can put together to satisfy all the core use cases. (Location 2205) #✂️

I call this approach composable design. A composable design does not aim to satisfy any use case in particular. You do not target any use case in particular not just because the use cases you were given were incomplete, faulty, and full of holes and contradictions, but because they will change. (Location 2214) #✂️

Since the goal of any system is to satisfy the requirements, composable design enables something else: design validation. Once you can produce an interaction between your services for each core use case, you have produced a valid design. (Location 2222) #✂️

Figure 4-1 is, in The Method’s parlance, a call chain diagram. (Location 2230) #✂️

A call chain demonstrates the interaction between components required to satisfy a particular use case. You can literally superimpose the call chain onto the layered architecture diagram. (Location 2233) #✂️

The components in the diagram are connected by arrows indicating the direction and type of the call between components—a solid black arrow for synchronous (request/response) calls, and a dashed gray arrow for a queued call. (Location 2234) #✂️

A sequence diagram in The Method’s parlance is similar to a UML sequence diagram.1 However, it includes notational differences to assure common meanings between diagram types. (Location 2242) #✂️

Remember: Your mission as the architect is to identify not just a set of components that you can put together to satisfy all the core use case, but the smallest set of components. (Location 2258) #✂️

The smallest set of services required in a typical software system contains 10 services in order of magnitude (e.g., sets of both 12 and 20 are on the order of 10). This particular order of magnitude is another universal design concept. (Location 2270) #✂️

Using The Method, even in a large system you are commonly looking at two to five Managers, two to three Engines, three to eight ResourceAccess and Resources, and a half-dozen Utilities. (Location 2276) #✂️

You may spend weeks or months trying to identify the core use cases and the areas of volatility. However, that is not design—that is requirements gathering and requirements analysis, which may be very time-consuming indeed. (Location 2286) #✂️

Features are always and everywhere aspects of integration, not implementation. (Location 2293) #✂️

The first order of business is to get all stakeholders to agree on a common vision. The vision must drive everything, from architecture to commitments. Everything that you do later has to serve that vision and be justified by it. Of course, this cuts both ways—which is why it is a good idea to start with the vision. If something does not serve the vision, then it often has to do with politics and other secondary or tertiary concerns. This provides you with an excellent way of repelling irrelevant demands that do not support the agreed-upon vision. (Location 2556) #✂️

A platform for building applications to support the TradeMe marketplace. (Location 2561) #✂️ #disagree

Vision → Objectives → Mission Statement → Architecture (Location 2593) #✂️

This is a reversal of the typical dynamics, in which the architect pleads with management to avoid functional decomposition. It is a lot easier to drive the correct architecture through the business by aligning the architecture with the business’s vision, its objectives, and the mission statement. (Location 2595) #✂️

Before you dive into the act of system design, ensure everyone is on the same page by compiling a short glossary of domain terminology. (Location 2602) #✂️

A good way of starting a glossary is to answer the four classic questions of “who,” “what,” “how,” and “where.” You answer the questions by examining the system overview, the use cases, and customer interview notes, if you have any. (Location 2604) #✂️

For TradeMe, the answers to the four questions were as follows: (Location 2606) #✂️

This does not preclude having additional subsystems or imply that these will necessarily be all the subsystems needed—you always decompose based on volatility, and if a “what” is not volatile, then it will not merit a component in the architecture. (Location 2619) #✂️

There is nothing wrong with suggesting certain areas of volatility, and then examining the resultant architecture. If the result produces a spiderweb of interactions or is asymmetric, then the design is unlikely to be good. (Location 2641) #✂️

The main reason for choosing a message bus is because it supports the most important operational concept of TradeMe: the Message Is the Application design pattern. When using this design pattern, the “application” is nowhere to be found. There is no collection of components or services that you can point to and identify as the application. Instead, the system comprises a loose collection of services that post and receive messages to one another (over a message bus, although that is secondary consideration). These messages are related to each other. (Location 2777) #✂️

The use of granular services integrated over a message bus with the Message Is the Application design pattern is one of the best ways of preparing the system for the future. By “preparing the system for the future,” I specifically refer to the next epoch in software engineering, the use of the actor model. (Location 2796) #✂️

For more on the actor model, see Juval Lowy, Actors: The Past and Future of Software Engineering (YouTube/IDesignIncTV, 2017). (Location 2809) #✂️

In many cases, a simpler design in which the Clients just queue up calls to the Managers would be a better fit for the development team. Always calibrate the architecture to the capability and maturity of the developers and management. After all, it is a lot easier to morph the architecture than it is to bend the organization. Once the organizational capabilities have matured, you can incorporate a full Message Is the Application pattern. (Location 2815) #✂️

A workflow Manager is a service that enables you to create, store, retrieve, and execute workflows. In theory, it is just another Manager. In practice, however, such Managers almost always utilize some sort of third-party workflow execution tool and workflow storage. For each Client call, the workflow Manager loads not just the correct workflow type but also a specific instance of it, with a particular state and context; executes the workflow; and persists it back to the workflow store. (Location 2824) #✂️

The project needs can be classified into five levels: physical, safety, repeatability, engineering, and technology. (Location 3065) #✂️

Physical. This is the lowest level in the project pyramid of needs, dealing with physical survival. Much as a person must have air, food, water, clothing, and shelter, a project must have a workplace (even a virtual one) and a viable business model. (Location 3066) #✂️

Safety. Once the physical needs are satisfied, the project must have adequate funding (often in the form of allocated resources) and enough time to complete the work. (Location 3070) #✂️

Repeatability. Project repeatability describes the development organization’s ability to deliver successfully time and again, and is the foundation for control and execution. It assures that if you plan for and commit to a certain schedule and cost, you will deliver on those commitments. (Location 3074) #✂️

Engineering. Once the repeatability of the project effort is secured, the software project can, for the first time, turn its attention to the enticing aspects of software engineering. This includes architecture and detail design, quality assurance activities such as root cause analysis and correction (on a systemic level), and preventive work using hardened operating procedures. (Location 3078) #✂️

Technology. At this level are development technology, tools, methodology, operating systems, and related hard-core technical aspects. (Location 3082) #✂️

Part 1 of this book stated a universal design rule: Features are always and everywhere aspects of integration, not implementation. As such, there are no features in any of the early services. At some point you will have integrated enough to start seeing features. I call that point the system. (Location 3121) #✂️

Never base progress reports on features. Always base progress reports on integration. (Location 3137) #✂️

A single architect is absolutely crucial for design integrity. You can extend this observation to the general rule that the only way to allow for design integrity is to have a single architect own the design. The opposite is also true: If no single person owns the design and can visualize it cover-to-cover, the system will not have design integrity. (Location 3168) #✂️

As vital as the architect is to the project, the architect cannot work in isolation. On day 1, the project must have a core team in place. The core team consists of three roles: project manager, product manager, and architect. (Location 3191) #✂️

The project manager. The job of the project manager is to shield the team from the organization. Most organizations, even small ones, create too much noise. If that noise makes its way into the development team, it can paralyze the team. A good project manager is like a firewall, blocking the noise, allowing only sanctioned communication through. (Location 3195) #✂️

The product manager. The product manager should encapsulate the customers. Customers are also a constant source of noise. The product manager acts as a proxy for the customers. For example, when the architect needs to clarify the required behaviors, the architect should not chase customers; instead, the product manager should provide the answers. (Location 3201) #✂️

The architect. The architect is the technical manager, acting as the design lead, the process lead, and the technical lead of the project. The architect not only designs the system, but also sees it through development. (Location 3205) #✂️

The core team designs the project in the fuzzy front end leading to development. The fuzzy front end is a general term1 in all technical projects referring to the very start of the project. The front end commences when someone has an idea about the project, and it concludes when developers start construction. (Location 3225) #✂️

The result of project design is a set of plans, not a single plan. (Location 3255) #✂️

Officially, the name of the meeting should be the Software Development Plan Review, or SDP review. It makes no difference if your process does not have an SDP review point: Just call a meeting (no manager can refuse a meeting request whose subject line is “Software Development Plan Review”). Once the desired option is identified, management must literally sign off on the SDP document. This document now becomes your project’s life insurance policy because, as long as you do not deviate from the plan’s parameters, there is no reason to cancel your project. This does require proper tracking (as described in Appendix A) and project management. (Location 3269) #✂️

For now, recognize that you should always assign services to developers in a 1:1 ratio. The 1:1 ratio does not mean that a developer works on only one service, but rather that if you do a cross-section of the team at any moment in time, you will see a developer working on one and only one service. (Location 3284) #✂️

Any other way of assigning services to developers will result in failure. Examples of the poor assignment options include: Multiple developers per service. The motivation for assigning two (or more) developers to one service is not a surplus of developers, but rather the desire to complete the work sooner. However, two people cannot really work on the same thing at the same time, so some subscheme must be used: (Location 3288) #✂️

Multiple services per developer. The option of assigning two (or more) services to a single developer is just as bad. Suppose two services, A and B, each estimated as a month of work, are assigned to a single developer, with the developer expected to finish both after a single month. Since the sum of work is two months, not only will the services be incomplete after one month, but finishing them will take much longer. While the developer is working on the A service, the developer is not working on the B service, causing the developers dependent on the B service to demand that the developer work on the B service. (Location 3302) #✂️

When assigning services (or activities such as UI development), try to maintain task continuity, a logical continuation between tasks assigned to each person. Often, such task assignments follow the service dependency graph. If service A depends on service B, then assign A to the developer of B. One advantage is that the A developer who is already familiar with B needs less ramp-up time. (Location 3353) #✂️

Effort estimation is how you try to answer the question of how long something will take. There are two types of estimations: individual activity estimation (estimating the effort for an activity assigned to a resource) and overall project estimation. The two types of estimations are unrelated, because the overall duration of the project is not the sum of effort across all activities divided by number of resources. (Location 3366) #✂️

Overestimation never works because of Parkinson’s law. (Location 3382) #✂️

Good estimations are accurate, but not precise. For example, consider an activity that actually took 13 days and had 2 estimations: 10 days or 23.8 days. While the second estimation is far more precise, clearly the first estimation is better because it is more accurate. (Location 3424) #✂️

Estimations also must match the tracking resolution. If the project manager tracks the project on a weekly basis, any estimation less than a week is pointless because it is smaller than the measurement resolution. (Location 3429) #✂️

Confronted with the uncertain, take these steps: Ask first for the order of magnitude: Is the activity more like a day, a week, a month, or a year? (Location 3444) #✂️

Make an explicit effort to list the areas of uncertainty in the project and focus on estimating them. Always break down large activities into smaller, more manageable activities to greatly increase the accuracy of the estimations. (Location 3449) #✂️

Invest in an exploratory discovery effort that will give insight into the nature of the problem and reduce the uncertainty. (Location 3450) #✂️

One estimation technique dealing specifically with high uncertainty is part of Program Evaluation and Review Technique (PERT).3 For every activity, you provide three estimations: the most optimistic, the most pessimistic, and the most likely. (Location 3453) #✂️

With even a modest degree of repeatability (see Figure 6-1), it is unlikely that you could deliver the project faster or slower than similar projects in the organization’s past. The dominant factor in throughput and efficiency is the organization’s nature, its own unique fingerprint of maturity, which is something that does not change overnight or between projects. If it took your company a year to deliver a similar project in the past, then it will take it a year in the future. (Location 3474) #✂️

Some tools even use Monte Carlo simulations to narrow down the range of the variables based on your project attributes or historical records. I have used such tools for decades, and they produce accurate results. (Location 3483) #✂️

The broadband estimation is my adaptation of the Wideband Delphi4 estimation technique. The broadband estimation uses multiple individual estimations to identify the average of the overall project estimation, then adds a band of estimations above and below it. You use the estimations outside the band to gain insight into the nature of the project and refine the estimations, repeating this process until the band and the project estimations converge. 4. Barry Boehm, Software Engineering Economics (Prentice Hall, 1981). (Location 3485) #✂️

You start the project design with the estimated duration of the individual activities in the project. Before you estimate individual activities, you must prepare a meticulous list of all activities in the project, both coding and noncoding activities alike. In a way, even that list of activities is an estimation of the actual set of activities, so the same rationale about reducing uncertainties holds true here. (Location 3518) #✂️

If you ask others to estimate an activity, you must maintain a correct estimation dialog with them. Never dictate duration by saying, “You have two weeks!” Not only is that based on nothing, but the owner of the activity also does not feel accountable to actually finish in two weeks. When people are unaccountable, progress and quality will be lacking. Avoid leading questions, such as “It is going to take two weeks, right?” While this is somewhat better than dictating the estimation, you now bias the other party toward your estimation. (Location 3533) #✂️

Critical path analysis is the single most important project design technique. However, you cannot perform this analysis without the following prerequisites: The system architecture. You must have the decomposition of the system into services and other building blocks such as Clients and Managers (Location 3542) #✂️

A list of all project activities. Your list must contain both coding and noncoding activities. (Location 3547) #✂️

Activity effort estimation. Have an accurate estimation of the effort for each activity in the list of activities. (Location 3550) #✂️

Services dependency tree. Use the call chains to identify the dependencies between the various services in the architecture. (Location 3551) #✂️

Activity dependencies. Beyond the dependencies between your services, you must compile a list of how all activities depend on other activities, coding and noncoding alike. (Location 3552) #✂️

Planning assumptions. You must know the resources available for the project or, more correctly, the staffing scenarios that your plan calls for. (Location 3554) #✂️

You can graphically arrange the activities in the project into a network diagram. The network diagram shows all activities in the project and their dependencies. You first derive the activity dependencies from the way the call chains propagate through the system. (Location 3558) #✂️

You should turn the diagram in Figure 7-4 into the detailed abstract chart shown in Figure 7-5. That chart now contains all activities, coding and noncoding alike, such as architecture and system testing. (Location 3573) #✂️

The time to get to an activity, or the time it takes to be ready to start working on the activity, is the maximum of time of all network paths leading to that activity. In a more formal manner, you calculate the time for completing activity i in the project with this recursive formula: where: Ti is the time for completing activity i. Ei is the effort estimation for activity i. n is the number of activities leading directly to activity i. (Location 3580) #✂️

By calculating the activity times, you can identify the longest possible path in the network of activities. In this context, the longest path means the path with greatest duration, not necessarily the one with the greatest number of activities. (Location 3606) #✂️

Based on the effort estimation for each activity and the dependencies, using the formula given earlier and starting from activity 17, the longest path in the network is shown in bold. That longest path in the network is called the critical path. (Location 3611) #✂️

With so few developers, you will paint yourself into a corner in which a developer on the critical path needs a noncritical activity that is simply not ready yet (such as activity 15 needing activity 11). This promotes a noncritical activity to a critical activity, in effect creating a new and longer critical path. I call this situation subcritical staffing. When the project goes subcritical, it will miss its deadline because the old critical path no longer applies. (Location 3659) #✂️

All noncritical activities have float, which is the amount of time you could delay completing them without delaying the project. Critical activities have no float (or more precisely, their float is zero) since any delay in these activities would delay the project. When you assign resources to the project, follow this rule: Always assign resources based on float. (Location 3691) #✂️

Together, the project network, the critical path, and the float analysis allow you to calculate the duration of the project as well as when each activity should start with respect to the project beginning. (Location 3760) #✂️

If you have several projects in the organization, then you could arrange them such that developers are always phasing out of one project while phasing into another. Working this way yields a hundreds of percent increase in productivity, the classic “doing much more with less.” (Location 3780) #✂️ #disagree

You produce a chart such as Figure 7-9 by first staffing the project, then listing all the dates of interest (unique dates when activities start and end) in chronological order. You then count how many resources are required for each category of resources in each time period between dates of interest. (Location 3797) #✂️

The efficiency of a project is the ratio between the sum of effort across all activities (assuming perfect utilization of people) and the actual project cost. (Location 3889) #✂️

The expected efficiency of a well-designed system, along with a properly designed and staffed project, ranges between 15% and 25%. (Location 3893) #✂️

The earned value at time t is the ratio between the sum of estimated duration of all activities completed by time t divided by the sum of the estimated durations of all activities. (Location 3930) #✂️

The earned value curve is a simple and easy way to answer the question: “Does the plan make sense?” If the planned earned value is a straight line, or it exhibits the issues of Figure 7-20 or Figure 7-22, the project is in danger. If it looks like a shallow S, then at least you have hope that the plan is sound and sensible. (Location 4014) #✂️

The project network acts as a logical representation of the project for planning purposes. The technique for analyzing the network is called the critical path method, although it has as much to do with the noncritical activities as it does with the critical ones. (Location 4038) #✂️

An activity in a software project is any task that requires both time and a resource. (Location 4047) #✂️

The project is a collection of related activities, and the network diagram captures these activities and their dependencies. In network diagrams, there is no notion of order of execution or concurrency between the activities. (Location 4048) #✂️

Consequently, you should avoid node diagrams and use arrow diagrams. The initial arrow diagram learning curve is more than offset by the benefits of having a concise, clear, clutter-free model of your project. (Location 4116) #✂️

An activity’s total float is by how much time you can delay the completion of that activity without delaying the project as a whole. (Location 4151) #✂️

An activity’s free float is by how much time you can delay the completion of that activity without disturbing any other activity in the project. (Location 4169) #✂️

As stated in Chapter 7, the safest and most efficient way to assign resources to activities is based on float—or, given the definition of this chapter, total float. This is the safest method because you address the riskier activities first, and it is the most efficient because you maximize the percentage of time for which the resources are utilized. (Location 4243) #✂️

As just stated, assigning resources based on float allows you to trade float for cost. You may be tempted to trade all the project’s float for lower cost, but that is rarely a good idea because a project with little float has less tolerance for delays. (Location 4289) #✂️

In general, the following techniques are possible in any software project and will accelerate the project as a whole: Assure quality. (Location 4306) #✂️

Employ test engineers. (Location 4314) #✂️

Add software testers. (Location 4322) #✂️

Invest in infrastructure. (Location 4327) #✂️

Improve development skills. (Location 4332) #✂️

Improve the process. (Location 4339) #✂️

Adopt and employ standards. (Location 4346) #✂️

Provide access to external experts. (Location 4350) #✂️

Engage in peer reviews. (Location 4355) #✂️

However, you can do two things to immediately accelerate the schedule—either work with better resources or find ways of working in parallel. (Location 4370) #✂️

Schedule compression means accomplishing the same objectives faster, often by doing more work to finish the task or the project sooner. You can use these two compression techniques in combination with each other or in isolation, on parts of the project, on the project as a whole, or on individual activities. Both compression techniques end up increasing the direct cost (defined later) of the project while reducing the schedule. (Location 4372) #✂️

Removing dependencies often involves investing in additional activities that enable the parallel work in the first place: Contract design. (Location 4405) #✂️

Emulators development. (Location 4409) #✂️

Simulators development. (Location 4411) #✂️

Repeated integration and testing. (Location 4414) #✂️

The Zen of Architects1 simply states that for the beginner architect, there are many options of doing pretty much anything. For the master architect, however, there are only a few good options, and typically only one. 1. https://en.wikipedia.org/wiki/Zen_Mind,_Beginner’s_Mind (Location 501) #✂️

The Method is a simple and effective analysis and design technique. You can express The Method as a formula: The Method = System Design + Project Design With system design, The Method lays out a way of breaking down a big system into small modular components. The Method offers guidelines for the structure, role, and semantics of the components and how these components should interact. The result is the architecture of the system. With project design, The Method helps you provide management with several options for building the system. Each option is some combination of schedule, cost, and risk. Each option also serves as the system assembly instructions, and it sets up the project for execution and tracking. (Location 529) #✂️

Project design is the second part of the book and is far more important for success than system design. (Location 535) #✂️

Design validation is critical because an organization should not risk having a team start developing against an inadequate architecture or developing a system the organization cannot afford to build. (Location 542) #✂️

Early validation of the design is imperative. (Location 551) #✂️

Ideally, one week into the project, you must know if the architecture is going to hold water (or not). (Location 552) #✂️

Note that I am referring here to the system design, the architecture, not the detailed design of the system. Detailed design produces for each component in the architecture the key implementation artifacts, such as interfaces, class hierarchies, and data contracts. Detailed design takes longer to produce, can be done during the project execution, and may change as the system is constructed or evolved. (Location 554) #✂️

Similarly, you must validate your project design. Running out of time or running over budget (or both) mid-project is simply unacceptable. (Location 558) #✂️

No project should fail because it did not have enough time or resources from the start. This book shows you how to accurately calculate the project duration and costs and how to drive educated decisions. (Location 563) #✂️

Using The Method, you can produce an entire system design in mere days, typically in three to five days, with project design taking similar time. (Location 565) #✂️

In general, design is not time-consuming (as opposed to implementation). Building architects charge hourly and often work only a week or two at most designing a house. Constructing a house from the architect’s design might take an agonizing two to three years of working with contractors, and yet the architect did not take long to produce the architecture. (Location 572) #✂️

The time crunch also helps avoid design gold plating. Parkinson’s law2 states that work always expands to fill the allotted time. Given 10 days to complete a design that could be completed in five days, the architect will likely work on the design for 10 days. (Location 575) #✂️

2. Cyril N. Parkinson, “Parkinson’s Law,” The Economist (November 19, 1955). (Location 579) #✂️

Analysis-paralysis is a predicament that occurs when someone (or a group) who is otherwise capable, clever, and even hardworking (as are most software architects) is stuck in a seemingly endless cycle of analysis, design, new revelations, and back to more analysis. (Location 582) #✂️

The main reason for the paralysis is being unaware of the design decision tree for both the system and the project. The design decision tree is a general concept that applies to all design tasks, not just in software engineering. The design of any complex entity is a collection of many smaller design decisions, arranged hierarchically in a tree-like structure. (Location 585) #✂️

Invariably, at some point, a downstream design decision will invalidate a prior decision; all decisions made in between these two points will be invalid. Designing this way is akin to performing a bubble sort of the design decision tree. Since bubble sort roughly involves as many operations as the square of the number of elements involved, the penalty is severe. (Location 591) #✂️

Only after you have designed the system is there any point in designing the project to build that system. (Location 602) #✂️

One of the most valuable techniques in pruning the decision tree is the application of constraints. As pointed by Dr. Fredrick Brooks,3 contrary to common wisdom or intuition, the worst design problem is a clean canvas. (Location 605) #✂️

3. Frederick P. Brooks Jr., The Design of Design: Essays from a Computer Scientist (Upper Saddle River, NJ: Addison-Wesley, 2010). (Location 612) #✂️

Software architecture is the high-level design and structure of the software system. (Location 654) #✂️

The act of identifying the constituent components of a system is called system decomposition. (Location 658) #✂️

In a modern system and in this book, services (as in service-orientation) are the most granular unit of the architecture. However, the technology used to implement the components and their details (such as interfaces, operations, and class hierarchies) are detailed design aspects, not system decomposition. (Location 662) #✂️

Functional decomposition decomposes a system into its building blocks based on the functionality of the system. (Location 670) #✂️

At the very least, functional decomposition couples services to the requirements because the services are a reflection of the requirements. Any change in the required functionality imposes a change on the functional services. (Location 675) #✂️

Because functional decomposition is also decomposition based on time (call A and then call B), it effectively precludes individual reuse of services. Suppose another system also needs a B service (such as Billing). Built into the fabric of B is the notion that it was called after an A and before a C service (such as first Invoicing, and only then Billing against an invoice, and finally Shipping). (Location 684) #✂️

One way of performing functional decomposition is to have as many services as there are variations of the functionalities. This decomposition leads to an explosion of services, since a decently sized system may have hundreds of functionalities. Not only do you have too many services, but these services often duplicate a lot of the common functionality, each customized to their case. (Location 702) #✂️

Another functional decomposition approach is to lump all possible ways of performing the operations into mega services. This leads to bloating in the size of the services, making them overly complex and impossible to maintain. (Location 706) #✂️

Functional decomposition, therefore, tends to make services either too big and too few or too small and too many. You often see both afflictions side by side in the same system. (Location 709) #✂️

Functional decomposition often leads to flattening of the system hierarchy. Since each service or building block is devoted to a specific functionality, someone must combine these discrete functionalities into a required behavior. That someone is often the client. (Location 714) #✂️

The hallmark of a bad design is when any change to the system affects the client. Ideally, the client and services should be able to evolve independently. (Location 732) #✂️

Yet, when designed as in Figure 2-1, you are forced to pollute the client with the business logic of sequencing, ordering, error compensation, and duration of the calls. (Location 734) #✂️

Cyclomatic complexity measures the number of independent paths through the code of a class or service. The more the internals are convoluted and coupled, the higher the cyclomatic complexity score. (Location 747) #✂️

Another problem with the decomposition of Figure 2-1 is that it requires multiple points of entry to the system. The client (or clients) needs to enter the system in three places: once for the A, then for the B, then for the C service. This means there are multiple places to worry about authentication, authorization, scalability, instance management, transaction propagation, identities, hosting, and so on. (Location 761) #✂️

The advantage of doing so is that you get to keep the clients simple and even asynchronous: the clients issue the call to the A service. The A service then calls B, and B calls C. The problem now is that the functional services are coupled to each other and to the order of the functional calls. For example, you can call the Billing service only after the Invoicing service but before the Shipping service. In the case of Figure 2-3, built into the A service is the knowledge that it needs to call the B service. The B service can be called only after the A service and before the C service. A change in the required ordering of the calls is likely to affect all services up and down the chain because their implementation will have to change to reflect the new required order. (Location 773) #✂️

Chaining functionality leads to bloated services. (Location 801) #✂️

Functional decomposition holds an almost irresistible allure. It looks like a simple and clear way of designing the system, requiring you to simply list the required functionalities and then create a component in your architecture for each. Functional decomposition (and its kin, the domain decomposition discussed later) is how most systems are designed. (Location 842) #✂️

At all costs, you must resist the temptations of functional decomposition. (Location 847) #✂️

Consider building a house functionally, as if it were a software system. You start by listing all the required functionalities of the house, such as cooking, playing, resting, sleeping, and so on. You then create an actual component in the architecture for each functionality, (Location 875) #✂️

The derision in these pages does not mean functional decomposition is a bad idea. Functional decomposition has a place—it is a decent requirements discovery technique. (Location 893) #✂️

domain decomposition: decomposing a system into building blocks based on the business domains, such as sales, engineering, accounting, and shipping. Sadly, domain decomposition such as Figure 2-7 shows is even worse than the functional decomposition of Figure 2-6 (Location 901) #✂️

The problem is that you can never deploy a single feature in isolation. There is no business value in Billing independent from Invoicing and Shipping. (Location 944) #✂️

A crucial flaw of both functional and domain decomposition has to do with testing. With such designs, the level of coupling and complexity is so high that the only kind of testing developers can do is unit testing. However, that does not make unit testing important, and it is merely another example of the streetlight effect1 (i.e., searching for something where it is easiest to look). 1. https://en.wikipedia.org/wiki/Streetlight_effect (Location 968) #✂️

Contrary to intuition, software requires design even more than physical systems do. The reason is simple: complexity. The complexity of physical systems such as typical houses is capped by physical constraints. (Location 996) #✂️

Without such natural physical restraints, complexity in software systems can get quickly out of control. The only way to rein in that complexity is to apply good engineering methods, of which design and process are paramount. (Location 1001) #✂️

Decompose based on volatility. Volatility-based decomposition identifies areas of potential change and encapsulates those into services or system building blocks. You then implement the required behavior as the interaction between the encapsulated areas of volatility. (Location 1091) #✂️

Functional decomposition therefore tends to maximize the effect of the change. Since most software systems are designed functionally, change is often painful and expensive, and the system is likely to resonate with the change. (Location 1103) #✂️

Accommodating change is the real reason you must avoid functional decomposition. (Location 1105) #✂️

A functional decomposition of your own body would have components for every task you are required to do, from driving to programming to presenting, yet your body does not have any such components. (Location 1139) #✂️

Volatility-based decomposition lends well to regression testing. The reduction in the number of components, the reduction in the size of components, and the simplification of the interactions between components all drastically reduce the complexity of the system. This makes it feasible to write regression testing that tests the system end to end, tests each subsystem individually, and eventually tests independent components. (Location 1157) #✂️

In 1972, David Parnas (an early pioneer of software engineering) published a seminal paper called “On the Criteria to Be Used in Decomposing Systems into Modules.”a This short, five-page paper contains most elements of modern software engineering, including encapsulation, information hiding, cohesion, modules, and loose coupling. (Location 1165) #✂️

It is the 2% problem again: it is not worth your while learning how to fix that sink if it is clogged less than 2% of the time. The moral is that when you spend 2% of your time on any complex task, you will never be any good at it. (Location 1199) #✂️

In 1999, David Dunning and Justin Kruger published their research2 demonstrating conclusively that people unskilled in a domain tend to look down on it, thinking it is less complex, risky, or demanding than it truly is. This cognitive bias has nothing to do with intelligence or expertise in other domains. If you are unskilled in something, you never assume it is more complex than it is, you assume it is less! 2. Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology 77, no. 6 (1999): 1121–1134. (Location 1215) #✂️

Not everything that is variable is also volatile. (Location 1237) #✂️

You resort to encapsulating a volatility at the system design level only when it is open-ended and, unless encapsulated in a component of the architecture, would be very expensive to contain. (Location 1237) #✂️

Variability, on the other hand, describes those aspects that you can easily handle in your code using conditional logic. (Location 1238) #✂️

When searching for volatility, you should be on the lookout for the kind of changes or risks that would have ripple effects across the system. (Location 1239) #✂️

There is a simple technique I call axes of volatility. This technique examines the ways the system is used by customers. (Location 1243) #✂️

Customer in this context refers to a consumer of the system, which could be a single user or a whole other business entity. (Location 1244) #✂️

In any business, there are only two ways your system could face change: the first axis is at the same customer over time. Even if presently the system is perfectly aligned with a particular customer’s needs, over time, that customer’s business context will change. (Location 1245) #✂️

The tendency of a solution to change the requirements against which it was developed was first observed by the 19th-century English economist William Jevons with regard to coal production, and it is referred to since as the Jevons paradox. Other manifestations are the increase in paper consumption with the digital office and the worsening traffic congestion following an increase in road capacity. (Location 1250) #✂️

The second way change could come is at the same time across customers. (Location 1253) #✂️

When searching for potential volatility in interviews, you will find it very helpful to phrase the questions in terms of the axes of volatility (same customer over time, all customers at the same point in time). (Location 1256) #✂️

Almost always, the axes should be independent. Something that changes for one customer over time should not change as much across all customers at the same point in time, and vice versa. If areas of change cannot be isolated to one of the axes, it often indicates a functional decomposition in disguise. (Location 1270) #✂️

Since most requirements specifications are chock-full of solutions masquerading as requirements, functional decomposition absolutely maximizes your pain. You will forever be chasing the ever-evolving solutions, never recognizing the true underlying requirements. (Location 1325) #✂️

The fact that requirements specifications have all those solutions masquerading as requirements is actually a blessing in disguise because you can generalize the example of cooking in the house into a bona fide analysis technique for discovering areas of volatility. (Location 1327) #✂️

Prior to decomposing a system and creating an architecture, you should simply compile a list of the candidate areas of volatility as a natural part of requirements gathering and analysis. (Location 1333) #✂️

During system decomposition, you must identify both the areas of volatility to encapsulate and those not to encapsulate (e.g., the nature of the business). Sometimes, you will have initial difficulty in telling these apart. There are two simple indicators if something that could change is indeed part of the nature of the business. The first indicator is that the possible change is rare. Yes, it could happen, but the likelihood of it happening is very low. The second indicator is that any attempt to encapsulate the change can only be done poorly. No practical amount of investment in time or effort will properly encapsulate the aspect in a way of which you can be proud. (Location 1470) #✂️

Another useful technique for identifying volatilities is to try to design a system for your competitor (or another division in your company). (Location 1508) #✂️

Ask yourself the following question: Can Federal Express use the software system UPS is using? Can UPS use the system Federal Express wants to build? If the likely answer is no, start listing all the barriers for such a reuse or extensibility. While both companies perform in the abstract the same service, the way they conduct their business is different. (Location 1511) #✂️

The opposite case is also true. If you and your competitor (and even better, all competitors) do some activity or sequence the same way, and there is no chance of your system doing it any other way, then there is no need to allocate a component in the architecture for that activity. To do so would create a functional decomposition. When you encounter something your competitors do identically, more likely than not, it represents the nature of the business, and as discussed previously, you should not encapsulate it. (Location 1518) #✂️

Volatility is intimately related to longevity. The longer the company or the application has been doing something the same way, the higher the likelihood the company will keep doing it the same way. Put differently, the longer things do not change, the longer they have until they do change or are replaced. You must put forward a design that accommodates such changes, even if at first glance such changes are independent of the current requirements. (Location 1523) #✂️

You can even guesstimate how long it will be until such a change is likely to take place using a simple heuristic: the ability of the organization (or the customer or the market) to instigate or absorb a change is more or less constant because it is tied to the nature of the business. (Location 1526) #✂️

Before diving into architecture, consider requirements. Most projects, if they even bother to capture the requirements, use functional requirements. Functional requirements simply state the required functionality, such as “The system should do A.” This is actually a poor way of specifying requirements, because it leaves the system’s implementation of the A functionality open for interpretation. (Location 1578) #✂️

Requirements should capture the required behavior rather than the required functionality. You should specify how the system is required to operate as opposed to what it should do, which is arguably the essence of requirements gathering. (Location 1585) #✂️

A use case is an expression of required behavior—that is, how the system is required to go about accomplishing some work and adding value to the business. As such, a use case is a particular sequence of activities in the system. Use cases tend to be verbose and descriptive. They can describe end-user interactions with the system, or the system’s interactions with other systems, or back-end processing. (Location 1589) #✂️

The best way of capturing a use case is graphically, with a diagram (Figure 3-1). Humans perform image processing astonishingly quickly, because almost half the human brain is a massive video processing unit. (Location 1599) #✂️

My rule of thumb: The presence of a nested “if” tells you that you should draw the use case. (Location 1605) #✂️

The Method prefers activity diagrams1 for graphical representation of use cases, primarily because activity diagrams can capture time-critical aspects of behavior, something that flowcharts and other diagrams are incapable of doing. (Location 1611) #✂️

Software systems are typically designed in layers, and The Method relies heavily on layers. Layers allow you to layer encapsulation. Each layer encapsulates its own volatilities from the layers above and the volatilities in the layers below. (Location 1626) #✂️

Even simple systems should be designed in layers to gain the benefit of encapsulation. In theory, the more layers, the better the encapsulation. Practical systems will have only a handful of layers, terminating with a layer of actual physical resources such as a data storage or a message queue. (Location 1631) #✂️

The preferred way of crossing layers is by calling services. While you certainly can benefit from the structure of The Method and volatility-based decomposition even with regular classes, relying on services provides distinct advantages. Which technology and platform you use to implement your services is a secondary concern. (Location 1634) #✂️

When you do use services (as long as the technology you chose allows), you immediately gain the following benefits: Scalability. (Location 1636) #✂️

Security. All service-oriented platforms treat security as a first-class aspect. (Location 1640) #✂️

Throughput and availability. Services can accept calls over queues, allowing you to handle a very large volume of messages by simply queuing up the excess load. (Location 1642) #✂️

Responsiveness. Services can throttle the calls into a buffer to avoid maxing out the system. (Location 1644) #✂️

Reliability. Clients and services can use some reliable messaging protocol to guarantee delivery, handle network connectivity issues, and even order the calls. (Location 1645) #✂️

Consistency. The services can all participate in the same unit of work, either in a transaction (when supported by the infrastructure) or in a coordinated business transaction that is eventually consistent. (Location 1646) #✂️

Synchronization. The calls to the service can be automatically synchronized even if the clients use multiple concurrent threads. (Location 1649) #✂️

The Method calls for four layers in the system architecture. These layers conform to some classic software engineering practices. (Location 1651) #✂️

The top layer in architecture is the client layer, also known as the presentation layer. I find the term “presentation” to be somewhat misleading. “Presentation” implies some information is being presented to human users, as if that is all that is expected from the top layer. The elements in the client layer may very well be end-user applications, but they can also be other systems interacting with your system. (Location 1656) #✂️

The client layer also encapsulates the potential volatility in Clients. Your system now and in the future across the axes of volatility may have different Clients such as desktop applications, web portals, mobile apps, holograms and augmented reality, APIs, administration applications, and so on. (Location 1667) #✂️

The business logic layer encapsulates the volatility in the system’s business logic. This layer implements the system’s required behavior, which, as mentioned previously, is best expressed in use cases. (Location 1673) #✂️

Use cases, however, are volatile, across both customers and time. Since a use case contains a sequence of activities in the system, a particular use case can change in only two ways: Either the sequence itself changes or the activities within the use case change. (Location 1675) #✂️

Both the sequence and the activities are volatile, and in The Method these volatilities are encapsulated in specific components called Managers and Engines. Manager components encapsulate the volatility in the sequence, whereas Engine components encapsulate the volatility in the activity. (Location 1687) #✂️

Since use cases are often related, Managers tend to encapsulate a family of logically related use cases, such as those in a particular subsystem. (Location 1693) #✂️

Since you can have great volatility in the sequence without any volatility in the activities of the sequence (see Figure 3-5), Managers may use zero or more Engines. Engines may be shared between Managers because you could perform an activity in one use case on behalf of one Manager and then perform the same activity for another Manager in a separate use case. (Location 1698) #✂️

You should design Engines with reuse in mind. However, if two Managers use two different Engines to perform the same activity, you either have functional decomposition on your hands or you have missed some activity volatility. (Location 1701) #✂️

The aptly named resource access layer encapsulates the volatility in accessing a resource, and the components in this layer are called ResourceAccess. (Location 1704) #✂️

While the motivation behind the resource access layer is readily evident and many systems incorporate some form of an access layer, most such layers end up exposing the underlying volatility by creating a ResourceAccess contract that resembles I/O operations or that is CRUD-like. For example, if your ResourceAccess service contract contains operations such as Select(), Insert(), and Delete(), the underlying resource is most likely a database. If you later change the database to a distributed cloud-based hash table, that database-access-like contract will become useless, and a new contract is required. Changing the contract affects every Engine and Manager that has used the ResourceAccess component. (Location 1710) #✂️

A well-designed ResourceAccess component exposes in its contract the atomic business verbs around a resource. (Location 1721) #✂️

The Method refers to these indivisible activities as atomic business verbs. (Location 1725) #✂️

Atomic business verbs are practically immutable because they relate strongly to the nature of the business, which, as discussed in Chapter 2, hardly ever changes. (Location 1728) #✂️

ResourceAccess services can be shared between Managers and Engines. You should explicitly design ResourceAccess components with this reuse in mind. If two Managers or two Engines cannot use the same ResourceAccess service when accessing the same resource or have some need for specific access, perhaps you did not encapsulate some access volatility or did not isolate the atomic business verbs correctly. (Location 1734) #✂️

The resource layer contains the actual physical Resources on which the system relies, such as a database, file system, a cache, or a message queue. In The Method, the Resource can be internal to the system or outside the system. Often, the Resource is a whole system in its own right, but to your system it appears as just a Resource. (Location 1738) #✂️

The Method recommends the following conventions for naming them: Names of services must be two-part compound words written in Pascal case. The suffix of the name is always the service’s type—for example, Manager, Engine, or Access (for ResourceAccess). The prefix varies with the type of service. –  For Managers, the prefix should be a noun associated with the encapsulated volatility in the use cases. –  For Engines, the prefix should be a noun describing the encapsulated activity. –  For ResourceAccess, the prefix should be a noun associated with the Resource, such as data that the service provides to the consuming use cases. Gerunds (a gerund is a noun created by tacking “ing” onto a verb) should be used as a prefix only in with Engines. The use of gerunds elsewhere in the business or access layers usually signals functional decomposition. Atomic business verbs should not be used in a prefix for a service name. These verbs should be confined to operation names in contracts interfacing with the resource access layer. (Location 1757) #✂️

The layers of services and resources in the architecture loosely correspond to the four English questions of “who,” “what,” “how,” and “where.” “Who” interacts with the system is in the Clients, “what” is required of the system is in Managers, “how” the system performs business activities is in Engines, “how” the system accesses Resources is in ResourceAccess, and “where” the system state is in Resources (see Figure 3-7). (Location 1778) #✂️

If your design contains a large number of Engines, you may have inadvertently done a functional decomposition. (Location 1803) #✂️

If your system has eight Managers, then you have already failed to produce a good design: The large number of Managers strongly indicates you have done a functional or domain decomposition. (Location 1809) #✂️

In a well-designed system, volatility should decrease top-down across the layers. Clients are very volatile. (Location 1821) #✂️

Managers do change, but not as much as their Clients. Managers change when the use cases—the required behavior of the system—change. (Location 1824) #✂️

Engines are less volatile than Managers. For an Engine to change, your business must change the way it is performing some activity, which is more uncommon than changing the sequencing of activities. (Location 1825) #✂️

ResourceAccess services are even less volatile than Engines. How often do you change the way you access a Resource or, for that matter, change the Resource? (Location 1826) #✂️

Resources are the least volatile components, changing at a glacial pace compared with the rest of the system. (Location 1829) #✂️

Reuse, unlike volatility, should increase going down the layers. (Location 1833) #✂️

Clients are hardly ever reusable. A Client application is typically developed for a particular type of platform and market and cannot be reused. (Location 1833) #✂️

Managers are reusable because you can use the same Manager and use cases from multiple Clients. (Location 1835) #✂️

Engines are even more reusable than Managers because the same Engine could be called by multiple Managers, in different use cases, to perform the same activity. (Location 1836) #✂️

ResourceAccess components are very reusable because they can be called by Engines and Managers. (Location 1838) #✂️

The Resources are the most reusable element in any well-designed systems. (Location 1838) #✂️

Managers can fall into one of three categories: expensive, expendable, and almost expendable. You can distinguish the category to which a Manager belongs by the way you respond when you are asked to change it. (Location 1841) #✂️

If your response is to fight the change, to fear its cost, to argue against the change, and so forth, then the Manager was clearly expensive and not expendable. An expensive Manager indicates that the Manager is too big, likely due to functional decomposition. If your response to the change request is just to shrug it off, thinking little of it, the Manager is pass-through and expendable. (Location 1843) #✂️

Expendable Managers are always a design flaw and a distortion of the architecture. They often exist only to satisfy the design guidelines without any real need for encapsulating use case volatility. (Location 1846) #✂️

A well-designed Manager service should be almost expendable. (Location 1851) #✂️

The Managers, Engines, and ResourceAccess are all services on their own right. A cohesive interaction between the Manager, Engines, and ResourceAccess may constitute a single logical service to external consumers. You can view such a set of interacting services as a logical subsystem. You group these together as a vertical slice of your system (Figure 3-8), where each vertical slice implements a corresponding set of use cases. (Location 1853) #✂️

Avoid over-partitioning your system into subsystems. Most systems should have only a handful of subsystems. Likewise, you should limit the number of Managers per subsystem to three. (Location 1858) #✂️

Design iteratively, build incrementally. (Location 1870) #✂️

There are two reasons why you can build only incrementally, and not iteratively. First, building iteratively is horrendously wasteful and difficult (turning a motorcycle into a car is much more difficult than just building a car). Second, and much more importantly, the intermediate iterations do not have any business value. If the customer wants a car to take the kids to school, what would the customer do with a motorcycle and why should the customer pay for it? (Location 1885) #✂️

The vertical slices of the system also enable you to accommodate extensibility. The correct way of extending any system is not by opening it up and hammering on existing components. (Location 1898) #✂️

In common usage, microservices correspond to domains or subsystems—that is, to the slices (red boxes) of Figure 3-8. There are three problems with this idea as practiced today. (Location 1923) #✂️

The first problem is the implied constraint on the number of services. If smaller services are better than larger services, why stop at the subsystem level? (Location 1925) #✂️

The second problem is the widespread use of functional decomposition in microservice design by the industry at large. (Location 1932) #✂️

The third problem relates to communication protocols. Although the choice of communication protocols has more to do with detailed design than with architecture, the effect of the choice is worth a passing comment here. (Location 1937) #✂️

As a general principle, in any well-designed system you should never use the same communication mechanism both internally and externally. (Location 1941) #✂️

Any layered architecture can have one of two possible operational models: open or closed. (Location 1959) #✂️

In an open architecture, any component can call any other component regardless of the layer in which the components reside. (Location 1962) #✂️

Calling sideways in this way is almost always the result of functional decomposition at the Managers level. (Location 1981) #✂️

When using open architecture, there is hardly any benefit of having architectural layers in the first place. In general, in software engineering, trading encapsulation for flexibility is a bad trade. (Location 1986) #✂️

In a closed architecture, you strive to maximize the benefits of the layers by disallowing calling up between layers and sideways within layers. (Location 1988) #✂️

Closed architecture promotes decoupling by trading flexibility for encapsulation. In general, that is a better trade than the other way around. (Location 1992) #✂️

A semi-closed/semi-open architecture allows calling more than one layer down. This, again, is a trade of encapsulation for flexibility and performance and, in general, is a trade to avoid. (Location 1997) #✂️

Notably, the use of semi-closed/semi-open architecture is justified in two classic cases. This first case occurs when you design some key piece of infrastructure, and you must squeeze every ounce of performance from it. (Location 1999) #✂️

The second case occurs within a codebase that hardly ever changes. (Location 2004) #✂️

In a closed architecture, Utilities pose a challenge. Consider Logging, a service used for recording run-time events. If you classify Logging as a Resource, then the ResourceAccess can use it, but the Managers cannot. If you place Logging at the same level as the Managers, only the Clients can log. The same goes for Security or Diagnostics—services that almost all other components require. In short, there is no good location for Utilities among the layers of a closed architecture. The Method places Utilities in a vertical bar on the side of the layers (see Figure 3-4). This bar cuts across all layers, allowing any component in the architecture to use any Utility. (Location 2016) #✂️

To qualify as a Utility, the component must pass a simple litmus test: Can the component plausibly be used in any other system, such as a smart cappuccino machine? (Location 2026) #✂️

This next guideline may be implied, but it is important enough to state explicitly. Because they are in the same layer, both Managers and Engines can call ResourceAccess services without violating the closed architecture (see Figure 3-4). Allowing Managers to call ResourceAccess is also implied from the section defining Managers and Engines. A Manager that uses no Engines must be able to access the underlying Resources. (Location 2033) #✂️

Managers can directly call Engines. The separation between Managers and Engines is almost at the detailed design level. Engines are really just an expression of the Strategy design pattern6 used to implement the activities within the Managers’ workflows. Therefore, Manager-to-Engine calls are not truly sideways calls, as is the case with Manager-to-Manager calls. (Location 2038) #✂️

While Managers should not call directly sideways to other Managers, a Manager can queue a call to another Manager. (Location 2046) #✂️

Using The Method structure, when a Manager queues a call to another Manager, the proxy is a ResourceAccess to the underlying Resource, the queue; that is, the call actually goes down, not sideways. (Location 2051) #✂️

The semantic explanation involves the nature of use cases. Business systems quite commonly have one use case that triggers a latent, much-deferred execution of another use case. For example, imagine a system in which a Manager executing a use case must save some system state for analysis at the end of the month. Without interrupting its flow, the Manager could queue the analysis request to another Manager. The second Manager could dequeue at the month’s end and perform its analysis workflow. The two use cases are independent and decoupled on the timeline. (Location 2054) #✂️

If you do one of the things on this list, you will likely live to regret it. Treat any violation of these rules as a red flag and investigate further to see what you are missing: (Location 2078) #✂️

Another universal design rule is that all good architectures are symmetric. (Location 2119) #✂️

The quest for symmetry, however, is only at the architecture level, not in detailed design. (Location 2121) #✂️

The symmetry in software systems manifests in repeated call patterns across use cases. You should expect symmetry, and its absence is a cause for concern. (Location 2124) #✂️

For example, suppose a Manager implements four use cases, three of which publish an event with the Pub/Sub service and the fourth of which does not. That break of symmetry is a design smell. Why is the fourth case different? What are you missing or overdoing? Is that Manager a real Manager, or is it a functionally decomposed component without volatility? Symmetry can also be broken by the presence of something, not just by its absence. (Location 2125) #✂️

But how do you know the composition of these components at run-time adequately satisfies all the requirements? (Location 2135) #✂️

Design validation and composition, as you will see, are intimately related. (Location 2137) #✂️

Never design against the requirements. (Location 2157) #✂️

As discussed in Chapter 3, the correct way of capturing the requirements is in the form of use cases: the required set of behaviors of the system. (Location 2166) #✂️

The core use cases represent the essence of the business of the system. As discussed in Chapter 2, the nature of the business hardly ever changes, and the same goes for the core use cases. (Location 2187) #✂️

Most systems have as few as two or three core use cases, and the number seldom exceeds six. (Location 2193) #✂️

A core use case will almost always be some kind of an abstraction of other use cases, and it may even require a new term or name to differentiate it from the rest. (Location 2199) #✂️

The whole point of requirements analysis is to recognize the core use cases (and the areas of volatility). (Location 2202) #✂️

Your mission as an architect is to identify the smallest set of components that you can put together to satisfy all the core use cases. (Location 2205) #✂️

I call this approach composable design. A composable design does not aim to satisfy any use case in particular. You do not target any use case in particular not just because the use cases you were given were incomplete, faulty, and full of holes and contradictions, but because they will change. (Location 2214) #✂️

Since the goal of any system is to satisfy the requirements, composable design enables something else: design validation. Once you can produce an interaction between your services for each core use case, you have produced a valid design. (Location 2222) #✂️

Figure 4-1 is, in The Method’s parlance, a call chain diagram. (Location 2230) #✂️

A call chain demonstrates the interaction between components required to satisfy a particular use case. You can literally superimpose the call chain onto the layered architecture diagram. (Location 2233) #✂️

The components in the diagram are connected by arrows indicating the direction and type of the call between components—a solid black arrow for synchronous (request/response) calls, and a dashed gray arrow for a queued call. (Location 2234) #✂️

A sequence diagram in The Method’s parlance is similar to a UML sequence diagram.1 However, it includes notational differences to assure common meanings between diagram types. (Location 2242) #✂️

Remember: Your mission as the architect is to identify not just a set of components that you can put together to satisfy all the core use case, but the smallest set of components. (Location 2258) #✂️

The smallest set of services required in a typical software system contains 10 services in order of magnitude (e.g., sets of both 12 and 20 are on the order of 10). This particular order of magnitude is another universal design concept. (Location 2270) #✂️

Using The Method, even in a large system you are commonly looking at two to five Managers, two to three Engines, three to eight ResourceAccess and Resources, and a half-dozen Utilities. (Location 2276) #✂️

You may spend weeks or months trying to identify the core use cases and the areas of volatility. However, that is not design—that is requirements gathering and requirements analysis, which may be very time-consuming indeed. (Location 2286) #✂️

Features are always and everywhere aspects of integration, not implementation. (Location 2293) #✂️

The first order of business is to get all stakeholders to agree on a common vision. The vision must drive everything, from architecture to commitments. Everything that you do later has to serve that vision and be justified by it. Of course, this cuts both ways—which is why it is a good idea to start with the vision. If something does not serve the vision, then it often has to do with politics and other secondary or tertiary concerns. This provides you with an excellent way of repelling irrelevant demands that do not support the agreed-upon vision. (Location 2556) #✂️

A platform for building applications to support the TradeMe marketplace. (Location 2561) #✂️ #disagree

Vision → Objectives → Mission Statement → Architecture (Location 2593) #✂️

This is a reversal of the typical dynamics, in which the architect pleads with management to avoid functional decomposition. It is a lot easier to drive the correct architecture through the business by aligning the architecture with the business’s vision, its objectives, and the mission statement. (Location 2595) #✂️

Before you dive into the act of system design, ensure everyone is on the same page by compiling a short glossary of domain terminology. (Location 2602) #✂️

A good way of starting a glossary is to answer the four classic questions of “who,” “what,” “how,” and “where.” You answer the questions by examining the system overview, the use cases, and customer interview notes, if you have any. (Location 2604) #✂️

For TradeMe, the answers to the four questions were as follows: (Location 2606) #✂️

This does not preclude having additional subsystems or imply that these will necessarily be all the subsystems needed—you always decompose based on volatility, and if a “what” is not volatile, then it will not merit a component in the architecture. (Location 2619) #✂️

There is nothing wrong with suggesting certain areas of volatility, and then examining the resultant architecture. If the result produces a spiderweb of interactions or is asymmetric, then the design is unlikely to be good. (Location 2641) #✂️

The main reason for choosing a message bus is because it supports the most important operational concept of TradeMe: the Message Is the Application design pattern. When using this design pattern, the “application” is nowhere to be found. There is no collection of components or services that you can point to and identify as the application. Instead, the system comprises a loose collection of services that post and receive messages to one another (over a message bus, although that is secondary consideration). These messages are related to each other. (Location 2777) #✂️

The use of granular services integrated over a message bus with the Message Is the Application design pattern is one of the best ways of preparing the system for the future. By “preparing the system for the future,” I specifically refer to the next epoch in software engineering, the use of the actor model. (Location 2796) #✂️

For more on the actor model, see Juval Lowy, Actors: The Past and Future of Software Engineering (YouTube/IDesignIncTV, 2017). (Location 2809) #✂️

In many cases, a simpler design in which the Clients just queue up calls to the Managers would be a better fit for the development team. Always calibrate the architecture to the capability and maturity of the developers and management. After all, it is a lot easier to morph the architecture than it is to bend the organization. Once the organizational capabilities have matured, you can incorporate a full Message Is the Application pattern. (Location 2815) #✂️

A workflow Manager is a service that enables you to create, store, retrieve, and execute workflows. In theory, it is just another Manager. In practice, however, such Managers almost always utilize some sort of third-party workflow execution tool and workflow storage. For each Client call, the workflow Manager loads not just the correct workflow type but also a specific instance of it, with a particular state and context; executes the workflow; and persists it back to the workflow store. (Location 2824) #✂️

The project needs can be classified into five levels: physical, safety, repeatability, engineering, and technology. (Location 3065) #✂️

Physical. This is the lowest level in the project pyramid of needs, dealing with physical survival. Much as a person must have air, food, water, clothing, and shelter, a project must have a workplace (even a virtual one) and a viable business model. (Location 3066) #✂️

Safety. Once the physical needs are satisfied, the project must have adequate funding (often in the form of allocated resources) and enough time to complete the work. (Location 3070) #✂️

Repeatability. Project repeatability describes the development organization’s ability to deliver successfully time and again, and is the foundation for control and execution. It assures that if you plan for and commit to a certain schedule and cost, you will deliver on those commitments. (Location 3074) #✂️

Engineering. Once the repeatability of the project effort is secured, the software project can, for the first time, turn its attention to the enticing aspects of software engineering. This includes architecture and detail design, quality assurance activities such as root cause analysis and correction (on a systemic level), and preventive work using hardened operating procedures. (Location 3078) #✂️

Technology. At this level are development technology, tools, methodology, operating systems, and related hard-core technical aspects. (Location 3082) #✂️

Part 1 of this book stated a universal design rule: Features are always and everywhere aspects of integration, not implementation. As such, there are no features in any of the early services. At some point you will have integrated enough to start seeing features. I call that point the system. (Location 3121) #✂️

Never base progress reports on features. Always base progress reports on integration. (Location 3137) #✂️

A single architect is absolutely crucial for design integrity. You can extend this observation to the general rule that the only way to allow for design integrity is to have a single architect own the design. The opposite is also true: If no single person owns the design and can visualize it cover-to-cover, the system will not have design integrity. (Location 3168) #✂️

As vital as the architect is to the project, the architect cannot work in isolation. On day 1, the project must have a core team in place. The core team consists of three roles: project manager, product manager, and architect. (Location 3191) #✂️

The project manager. The job of the project manager is to shield the team from the organization. Most organizations, even small ones, create too much noise. If that noise makes its way into the development team, it can paralyze the team. A good project manager is like a firewall, blocking the noise, allowing only sanctioned communication through. (Location 3195) #✂️

The product manager. The product manager should encapsulate the customers. Customers are also a constant source of noise. The product manager acts as a proxy for the customers. For example, when the architect needs to clarify the required behaviors, the architect should not chase customers; instead, the product manager should provide the answers. (Location 3201) #✂️

The architect. The architect is the technical manager, acting as the design lead, the process lead, and the technical lead of the project. The architect not only designs the system, but also sees it through development. (Location 3205) #✂️

The core team designs the project in the fuzzy front end leading to development. The fuzzy front end is a general term1 in all technical projects referring to the very start of the project. The front end commences when someone has an idea about the project, and it concludes when developers start construction. (Location 3225) #✂️

The result of project design is a set of plans, not a single plan. (Location 3255) #✂️

Officially, the name of the meeting should be the Software Development Plan Review, or SDP review. It makes no difference if your process does not have an SDP review point: Just call a meeting (no manager can refuse a meeting request whose subject line is “Software Development Plan Review”). Once the desired option is identified, management must literally sign off on the SDP document. This document now becomes your project’s life insurance policy because, as long as you do not deviate from the plan’s parameters, there is no reason to cancel your project. This does require proper tracking (as described in Appendix A) and project management. (Location 3269) #✂️

For now, recognize that you should always assign services to developers in a 1:1 ratio. The 1:1 ratio does not mean that a developer works on only one service, but rather that if you do a cross-section of the team at any moment in time, you will see a developer working on one and only one service. (Location 3284) #✂️

Any other way of assigning services to developers will result in failure. Examples of the poor assignment options include: Multiple developers per service. The motivation for assigning two (or more) developers to one service is not a surplus of developers, but rather the desire to complete the work sooner. However, two people cannot really work on the same thing at the same time, so some subscheme must be used: (Location 3288) #✂️

Multiple services per developer. The option of assigning two (or more) services to a single developer is just as bad. Suppose two services, A and B, each estimated as a month of work, are assigned to a single developer, with the developer expected to finish both after a single month. Since the sum of work is two months, not only will the services be incomplete after one month, but finishing them will take much longer. While the developer is working on the A service, the developer is not working on the B service, causing the developers dependent on the B service to demand that the developer work on the B service. (Location 3302) #✂️

When assigning services (or activities such as UI development), try to maintain task continuity, a logical continuation between tasks assigned to each person. Often, such task assignments follow the service dependency graph. If service A depends on service B, then assign A to the developer of B. One advantage is that the A developer who is already familiar with B needs less ramp-up time. (Location 3353) #✂️

Effort estimation is how you try to answer the question of how long something will take. There are two types of estimations: individual activity estimation (estimating the effort for an activity assigned to a resource) and overall project estimation. The two types of estimations are unrelated, because the overall duration of the project is not the sum of effort across all activities divided by number of resources. (Location 3366) #✂️

Overestimation never works because of Parkinson’s law. (Location 3382) #✂️

Good estimations are accurate, but not precise. For example, consider an activity that actually took 13 days and had 2 estimations: 10 days or 23.8 days. While the second estimation is far more precise, clearly the first estimation is better because it is more accurate. (Location 3424) #✂️

Estimations also must match the tracking resolution. If the project manager tracks the project on a weekly basis, any estimation less than a week is pointless because it is smaller than the measurement resolution. (Location 3429) #✂️

Confronted with the uncertain, take these steps: Ask first for the order of magnitude: Is the activity more like a day, a week, a month, or a year? (Location 3444) #✂️

Make an explicit effort to list the areas of uncertainty in the project and focus on estimating them. Always break down large activities into smaller, more manageable activities to greatly increase the accuracy of the estimations. (Location 3449) #✂️

Invest in an exploratory discovery effort that will give insight into the nature of the problem and reduce the uncertainty. (Location 3450) #✂️

One estimation technique dealing specifically with high uncertainty is part of Program Evaluation and Review Technique (PERT).3 For every activity, you provide three estimations: the most optimistic, the most pessimistic, and the most likely. (Location 3453) #✂️

With even a modest degree of repeatability (see Figure 6-1), it is unlikely that you could deliver the project faster or slower than similar projects in the organization’s past. The dominant factor in throughput and efficiency is the organization’s nature, its own unique fingerprint of maturity, which is something that does not change overnight or between projects. If it took your company a year to deliver a similar project in the past, then it will take it a year in the future. (Location 3474) #✂️

Some tools even use Monte Carlo simulations to narrow down the range of the variables based on your project attributes or historical records. I have used such tools for decades, and they produce accurate results. (Location 3483) #✂️

The broadband estimation is my adaptation of the Wideband Delphi4 estimation technique. The broadband estimation uses multiple individual estimations to identify the average of the overall project estimation, then adds a band of estimations above and below it. You use the estimations outside the band to gain insight into the nature of the project and refine the estimations, repeating this process until the band and the project estimations converge. 4. Barry Boehm, Software Engineering Economics (Prentice Hall, 1981). (Location 3485) #✂️

You start the project design with the estimated duration of the individual activities in the project. Before you estimate individual activities, you must prepare a meticulous list of all activities in the project, both coding and noncoding activities alike. In a way, even that list of activities is an estimation of the actual set of activities, so the same rationale about reducing uncertainties holds true here. (Location 3518) #✂️

If you ask others to estimate an activity, you must maintain a correct estimation dialog with them. Never dictate duration by saying, “You have two weeks!” Not only is that based on nothing, but the owner of the activity also does not feel accountable to actually finish in two weeks. When people are unaccountable, progress and quality will be lacking. Avoid leading questions, such as “It is going to take two weeks, right?” While this is somewhat better than dictating the estimation, you now bias the other party toward your estimation. (Location 3533) #✂️

Critical path analysis is the single most important project design technique. However, you cannot perform this analysis without the following prerequisites: The system architecture. You must have the decomposition of the system into services and other building blocks such as Clients and Managers (Location 3542) #✂️

A list of all project activities. Your list must contain both coding and noncoding activities. (Location 3547) #✂️

Activity effort estimation. Have an accurate estimation of the effort for each activity in the list of activities. (Location 3550) #✂️

Services dependency tree. Use the call chains to identify the dependencies between the various services in the architecture. (Location 3551) #✂️

Activity dependencies. Beyond the dependencies between your services, you must compile a list of how all activities depend on other activities, coding and noncoding alike. (Location 3552) #✂️

Planning assumptions. You must know the resources available for the project or, more correctly, the staffing scenarios that your plan calls for. (Location 3554) #✂️

You can graphically arrange the activities in the project into a network diagram. The network diagram shows all activities in the project and their dependencies. You first derive the activity dependencies from the way the call chains propagate through the system. (Location 3558) #✂️

You should turn the diagram in Figure 7-4 into the detailed abstract chart shown in Figure 7-5. That chart now contains all activities, coding and noncoding alike, such as architecture and system testing. (Location 3573) #✂️

The time to get to an activity, or the time it takes to be ready to start working on the activity, is the maximum of time of all network paths leading to that activity. In a more formal manner, you calculate the time for completing activity i in the project with this recursive formula: where: Ti is the time for completing activity i. Ei is the effort estimation for activity i. n is the number of activities leading directly to activity i. (Location 3580) #✂️

By calculating the activity times, you can identify the longest possible path in the network of activities. In this context, the longest path means the path with greatest duration, not necessarily the one with the greatest number of activities. (Location 3606) #✂️

Based on the effort estimation for each activity and the dependencies, using the formula given earlier and starting from activity 17, the longest path in the network is shown in bold. That longest path in the network is called the critical path. (Location 3611) #✂️

With so few developers, you will paint yourself into a corner in which a developer on the critical path needs a noncritical activity that is simply not ready yet (such as activity 15 needing activity 11). This promotes a noncritical activity to a critical activity, in effect creating a new and longer critical path. I call this situation subcritical staffing. When the project goes subcritical, it will miss its deadline because the old critical path no longer applies. (Location 3659) #✂️

All noncritical activities have float, which is the amount of time you could delay completing them without delaying the project. Critical activities have no float (or more precisely, their float is zero) since any delay in these activities would delay the project. When you assign resources to the project, follow this rule: Always assign resources based on float. (Location 3691) #✂️

Together, the project network, the critical path, and the float analysis allow you to calculate the duration of the project as well as when each activity should start with respect to the project beginning. (Location 3760) #✂️

If you have several projects in the organization, then you could arrange them such that developers are always phasing out of one project while phasing into another. Working this way yields a hundreds of percent increase in productivity, the classic “doing much more with less.” (Location 3780) #✂️ #disagree

You produce a chart such as Figure 7-9 by first staffing the project, then listing all the dates of interest (unique dates when activities start and end) in chronological order. You then count how many resources are required for each category of resources in each time period between dates of interest. (Location 3797) #✂️

The efficiency of a project is the ratio between the sum of effort across all activities (assuming perfect utilization of people) and the actual project cost. (Location 3889) #✂️

The expected efficiency of a well-designed system, along with a properly designed and staffed project, ranges between 15% and 25%. (Location 3893) #✂️

The earned value at time t is the ratio between the sum of estimated duration of all activities completed by time t divided by the sum of the estimated durations of all activities. (Location 3930) #✂️

The earned value curve is a simple and easy way to answer the question: “Does the plan make sense?” If the planned earned value is a straight line, or it exhibits the issues of Figure 7-20 or Figure 7-22, the project is in danger. If it looks like a shallow S, then at least you have hope that the plan is sound and sensible. (Location 4014) #✂️

The project network acts as a logical representation of the project for planning purposes. The technique for analyzing the network is called the critical path method, although it has as much to do with the noncritical activities as it does with the critical ones. (Location 4038) #✂️

An activity in a software project is any task that requires both time and a resource. (Location 4047) #✂️

The project is a collection of related activities, and the network diagram captures these activities and their dependencies. In network diagrams, there is no notion of order of execution or concurrency between the activities. (Location 4048) #✂️

Consequently, you should avoid node diagrams and use arrow diagrams. The initial arrow diagram learning curve is more than offset by the benefits of having a concise, clear, clutter-free model of your project. (Location 4116) #✂️

An activity’s total float is by how much time you can delay the completion of that activity without delaying the project as a whole. (Location 4151) #✂️

An activity’s free float is by how much time you can delay the completion of that activity without disturbing any other activity in the project. (Location 4169) #✂️

As stated in Chapter 7, the safest and most efficient way to assign resources to activities is based on float—or, given the definition of this chapter, total float. This is the safest method because you address the riskier activities first, and it is the most efficient because you maximize the percentage of time for which the resources are utilized. (Location 4243) #✂️

As just stated, assigning resources based on float allows you to trade float for cost. You may be tempted to trade all the project’s float for lower cost, but that is rarely a good idea because a project with little float has less tolerance for delays. (Location 4289) #✂️

In general, the following techniques are possible in any software project and will accelerate the project as a whole: Assure quality. (Location 4306) #✂️

Employ test engineers. (Location 4314) #✂️

Add software testers. (Location 4322) #✂️

Invest in infrastructure. (Location 4327) #✂️

Improve development skills. (Location 4332) #✂️

Improve the process. (Location 4339) #✂️

Adopt and employ standards. (Location 4346) #✂️

Provide access to external experts. (Location 4350) #✂️

Engage in peer reviews. (Location 4355) #✂️

However, you can do two things to immediately accelerate the schedule—either work with better resources or find ways of working in parallel. (Location 4370) #✂️

Schedule compression means accomplishing the same objectives faster, often by doing more work to finish the task or the project sooner. You can use these two compression techniques in combination with each other or in isolation, on parts of the project, on the project as a whole, or on individual activities. Both compression techniques end up increasing the direct cost (defined later) of the project while reducing the schedule. (Location 4372) #✂️

Removing dependencies often involves investing in additional activities that enable the parallel work in the first place: Contract design. (Location 4405) #✂️

Emulators development. (Location 4409) #✂️

Simulators development. (Location 4411) #✂️

Repeated integration and testing. (Location 4414) #✂️