The Zen of Architects1 simply states that for the beginner architect, there are many options of doing pretty much anything. For the master architect, however, there are only a few good options, and typically only one. 1. https://en.wikipedia.org/wiki/Zen_Mind,_Beginner’s_Mind (501)

The Method is a simple and effective analysis and design technique. You can express The Method as a formula: The Method = System Design + Project Design With system design, The Method lays out a way of breaking down a big system into small modular components. The Method offers guidelines for the structure, role, and semantics of the components and how these components should interact. The result is the architecture of the system. With project design, The Method helps you provide management with several options for building the system. Each option is some combination of schedule, cost, and risk. Each option also serves as the system assembly instructions, and it sets up the project for execution and tracking. (529)

Project design is the second part of the book and is far more important for success than system design. (535)

Design validation is critical because an organization should not risk having a team start developing against an inadequate architecture or developing a system the organization cannot afford to build. (542)

Early validation of the design is imperative. (551)

Ideally, one week into the project, you must know if the architecture is going to hold water (or not). (552)

Note that I am referring here to the system design, the architecture, not the detailed design of the system. Detailed design produces for each component in the architecture the key implementation artifacts, such as interfaces, class hierarchies, and data contracts. Detailed design takes longer to produce, can be done during the project execution, and may change as the system is constructed or evolved. (554)

Similarly, you must validate your project design. Running out of time or running over budget (or both) mid-project is simply unacceptable. (558)

No project should fail because it did not have enough time or resources from the start. This book shows you how to accurately calculate the project duration and costs and how to drive educated decisions. (563)

Using The Method, you can produce an entire system design in mere days, typically in three to five days, with project design taking similar time. (565)

In general, design is not time-consuming (as opposed to implementation). Building architects charge hourly and often work only a week or two at most designing a house. Constructing a house from the architect’s design might take an agonizing two to three years of working with contractors, and yet the architect did not take long to produce the architecture. (572)

The time crunch also helps avoid design gold plating. Parkinson’s law2 states that work always expands to fill the allotted time. Given 10 days to complete a design that could be completed in five days, the architect will likely work on the design for 10 days. (575)

2. Cyril N. Parkinson, “Parkinson’s Law,” The Economist (November 19, 1955). (579)

Analysis-paralysis is a predicament that occurs when someone (or a group) who is otherwise capable, clever, and even hardworking (as are most software architects) is stuck in a seemingly endless cycle of analysis, design, new revelations, and back to more analysis. (582)

The main reason for the paralysis is being unaware of the design decision tree for both the system and the project. The design decision tree is a general concept that applies to all design tasks, not just in software engineering. The design of any complex entity is a collection of many smaller design decisions, arranged hierarchically in a tree-like structure. (585)

Invariably, at some point, a downstream design decision will invalidate a prior decision; all decisions made in between these two points will be invalid. Designing this way is akin to performing a bubble sort of the design decision tree. Since bubble sort roughly involves as many operations as the square of the number of elements involved, the penalty is severe. (591)

Only after you have designed the system is there any point in designing the project to build that system. (602)

One of the most valuable techniques in pruning the decision tree is the application of constraints. As pointed by Dr. Fredrick Brooks,3 contrary to common wisdom or intuition, the worst design problem is a clean canvas. (605)

3. Frederick P. Brooks Jr., The Design of Design: Essays from a Computer Scientist (Upper Saddle River, NJ: Addison-Wesley, 2010). (612)

Software architecture is the high-level design and structure of the software system. (654)

The act of identifying the constituent components of a system is called system decomposition. (658)

In a modern system and in this book, services (as in service-orientation) are the most granular unit of the architecture. However, the technology used to implement the components and their details (such as interfaces, operations, and class hierarchies) are detailed design aspects, not system decomposition. (662)

Functional decomposition decomposes a system into its building blocks based on the functionality of the system. (670)

At the very least, functional decomposition couples services to the requirements because the services are a reflection of the requirements. Any change in the required functionality imposes a change on the functional services. (675)

Because functional decomposition is also decomposition based on time (call A and then call B), it effectively precludes individual reuse of services. Suppose another system also needs a B service (such as Billing). Built into the fabric of B is the notion that it was called after an A and before a C service (such as first Invoicing, and only then Billing against an invoice, and finally Shipping). (684)

One way of performing functional decomposition is to have as many services as there are variations of the functionalities. This decomposition leads to an explosion of services, since a decently sized system may have hundreds of functionalities. Not only do you have too many services, but these services often duplicate a lot of the common functionality, each customized to their case. (702)

Another functional decomposition approach is to lump all possible ways of performing the operations into mega services. This leads to bloating in the size of the services, making them overly complex and impossible to maintain. (706)

Functional decomposition, therefore, tends to make services either too big and too few or too small and too many. You often see both afflictions side by side in the same system. (709)

Functional decomposition often leads to flattening of the system hierarchy. Since each service or building block is devoted to a specific functionality, someone must combine these discrete functionalities into a required behavior. That someone is often the client. (714)

The hallmark of a bad design is when any change to the system affects the client. Ideally, the client and services should be able to evolve independently. (732)

Yet, when designed as in Figure 2-1, you are forced to pollute the client with the business logic of sequencing, ordering, error compensation, and duration of the calls. (734)

Cyclomatic complexity measures the number of independent paths through the code of a class or service. The more the internals are convoluted and coupled, the higher the cyclomatic complexity score. (747)

Another problem with the decomposition of Figure 2-1 is that it requires multiple points of entry to the system. The client (or clients) needs to enter the system in three places: once for the A, then for the B, then for the C service. This means there are multiple places to worry about authentication, authorization, scalability, instance management, transaction propagation, identities, hosting, and so on. (761)

The advantage of doing so is that you get to keep the clients simple and even asynchronous: the clients issue the call to the A service. The A service then calls B, and B calls C. The problem now is that the functional services are coupled to each other and to the order of the functional calls. For example, you can call the Billing service only after the Invoicing service but before the Shipping service. In the case of Figure 2-3, built into the A service is the knowledge that it needs to call the B service. The B service can be called only after the A service and before the C service. A change in the required ordering of the calls is likely to affect all services up and down the chain because their implementation will have to change to reflect the new required order. (773)

Chaining functionality leads to bloated services. (801)

Functional decomposition holds an almost irresistible allure. It looks like a simple and clear way of designing the system, requiring you to simply list the required functionalities and then create a component in your architecture for each. Functional decomposition (and its kin, the domain decomposition discussed later) is how most systems are designed. (842)

At all costs, you must resist the temptations of functional decomposition. (847)

Consider building a house functionally, as if it were a software system. You start by listing all the required functionalities of the house, such as cooking, playing, resting, sleeping, and so on. You then create an actual component in the architecture for each functionality, (875)

The derision in these pages does not mean functional decomposition is a bad idea. Functional decomposition has a place—it is a decent requirements discovery technique. (893)

domain decomposition: decomposing a system into building blocks based on the business domains, such as sales, engineering, accounting, and shipping. Sadly, domain decomposition such as Figure 2-7 shows is even worse than the functional decomposition of Figure 2-6 (901)

The problem is that you can never deploy a single feature in isolation. There is no business value in Billing independent from Invoicing and Shipping. (944)

A crucial flaw of both functional and domain decomposition has to do with testing. With such designs, the level of coupling and complexity is so high that the only kind of testing developers can do is unit testing. However, that does not make unit testing important, and it is merely another example of the streetlight effect1 (i.e., searching for something where it is easiest to look). 1. https://en.wikipedia.org/wiki/Streetlight_effect (968)

Contrary to intuition, software requires design even more than physical systems do. The reason is simple: complexity. The complexity of physical systems such as typical houses is capped by physical constraints. (996)

Without such natural physical restraints, complexity in software systems can get quickly out of control. The only way to rein in that complexity is to apply good engineering methods, of which design and process are paramount. (1001)

Decompose based on volatility. Volatility-based decomposition identifies areas of potential change and encapsulates those into services or system building blocks. You then implement the required behavior as the interaction between the encapsulated areas of volatility. (1091)

Functional decomposition therefore tends to maximize the effect of the change. Since most software systems are designed functionally, change is often painful and expensive, and the system is likely to resonate with the change. (1103)

Accommodating change is the real reason you must avoid functional decomposition. (1105)

A functional decomposition of your own body would have components for every task you are required to do, from driving to programming to presenting, yet your body does not have any such components. (1139)

Volatility-based decomposition lends well to regression testing. The reduction in the number of components, the reduction in the size of components, and the simplification of the interactions between components all drastically reduce the complexity of the system. This makes it feasible to write regression testing that tests the system end to end, tests each subsystem individually, and eventually tests independent components. (1157)

In 1972, David Parnas (an early pioneer of software engineering) published a seminal paper called “On the Criteria to Be Used in Decomposing Systems into Modules.”a This short, five-page paper contains most elements of modern software engineering, including encapsulation, information hiding, cohesion, modules, and loose coupling. (1165)

It is the 2% problem again: it is not worth your while learning how to fix that sink if it is clogged less than 2% of the time. The moral is that when you spend 2% of your time on any complex task, you will never be any good at it. (1199)

In 1999, David Dunning and Justin Kruger published their research2 demonstrating conclusively that people unskilled in a domain tend to look down on it, thinking it is less complex, risky, or demanding than it truly is. This cognitive bias has nothing to do with intelligence or expertise in other domains. If you are unskilled in something, you never assume it is more complex than it is, you assume it is less! 2. Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology 77, no. 6 (1999): 1121–1134. (1215)

Not everything that is variable is also volatile. (1237)

You resort to encapsulating a volatility at the system design level only when it is open-ended and, unless encapsulated in a component of the architecture, would be very expensive to contain. (1237)

Variability, on the other hand, describes those aspects that you can easily handle in your code using conditional logic. (1238)

When searching for volatility, you should be on the lookout for the kind of changes or risks that would have ripple effects across the system. (1239)

There is a simple technique I call axes of volatility. This technique examines the ways the system is used by customers. (1243)

Customer in this context refers to a consumer of the system, which could be a single user or a whole other business entity. (1244)

In any business, there are only two ways your system could face change: the first axis is at the same customer over time. Even if presently the system is perfectly aligned with a particular customer’s needs, over time, that customer’s business context will change. (1245)

The tendency of a solution to change the requirements against which it was developed was first observed by the 19th-century English economist William Jevons with regard to coal production, and it is referred to since as the Jevons paradox. Other manifestations are the increase in paper consumption with the digital office and the worsening traffic congestion following an increase in road capacity. (1250)

The second way change could come is at the same time across customers. (1253)

When searching for potential volatility in interviews, you will find it very helpful to phrase the questions in terms of the axes of volatility (same customer over time, all customers at the same point in time). (1256)

Almost always, the axes should be independent. Something that changes for one customer over time should not change as much across all customers at the same point in time, and vice versa. If areas of change cannot be isolated to one of the axes, it often indicates a functional decomposition in disguise. (1270)

Since most requirements specifications are chock-full of solutions masquerading as requirements, functional decomposition absolutely maximizes your pain. You will forever be chasing the ever-evolving solutions, never recognizing the true underlying requirements. (1325)

The fact that requirements specifications have all those solutions masquerading as requirements is actually a blessing in disguise because you can generalize the example of cooking in the house into a bona fide analysis technique for discovering areas of volatility. (1327)

Prior to decomposing a system and creating an architecture, you should simply compile a list of the candidate areas of volatility as a natural part of requirements gathering and analysis. (1333)

During system decomposition, you must identify both the areas of volatility to encapsulate and those not to encapsulate (e.g., the nature of the business). Sometimes, you will have initial difficulty in telling these apart. There are two simple indicators if something that could change is indeed part of the nature of the business. The first indicator is that the possible change is rare. Yes, it could happen, but the likelihood of it happening is very low. The second indicator is that any attempt to encapsulate the change can only be done poorly. No practical amount of investment in time or effort will properly encapsulate the aspect in a way of which you can be proud. (1470)

Another useful technique for identifying volatilities is to try to design a system for your competitor (or another division in your company). (1508)

Ask yourself the following question: Can Federal Express use the software system UPS is using? Can UPS use the system Federal Express wants to build? If the likely answer is no, start listing all the barriers for such a reuse or extensibility. While both companies perform in the abstract the same service, the way they conduct their business is different. (1511)

The opposite case is also true. If you and your competitor (and even better, all competitors) do some activity or sequence the same way, and there is no chance of your system doing it any other way, then there is no need to allocate a component in the architecture for that activity. To do so would create a functional decomposition. When you encounter something your competitors do identically, more likely than not, it represents the nature of the business, and as discussed previously, you should not encapsulate it. (1518)

Volatility is intimately related to longevity. The longer the company or the application has been doing something the same way, the higher the likelihood the company will keep doing it the same way. Put differently, the longer things do not change, the longer they have until they do change or are replaced. You must put forward a design that accommodates such changes, even if at first glance such changes are independent of the current requirements. (1523)

You can even guesstimate how long it will be until such a change is likely to take place using a simple heuristic: the ability of the organization (or the customer or the market) to instigate or absorb a change is more or less constant because it is tied to the nature of the business. (1526)

Before diving into architecture, consider requirements. Most projects, if they even bother to capture the requirements, use functional requirements. Functional requirements simply state the required functionality, such as “The system should do A.” This is actually a poor way of specifying requirements, because it leaves the system’s implementation of the A functionality open for interpretation. (1578)

Requirements should capture the required behavior rather than the required functionality. You should specify how the system is required to operate as opposed to what it should do, which is arguably the essence of requirements gathering. (1585)

A use case is an expression of required behavior—that is, how the system is required to go about accomplishing some work and adding value to the business. As such, a use case is a particular sequence of activities in the system. Use cases tend to be verbose and descriptive. They can describe end-user interactions with the system, or the system’s interactions with other systems, or back-end processing. (1589)

The best way of capturing a use case is graphically, with a diagram (Figure 3-1). Humans perform image processing astonishingly quickly, because almost half the human brain is a massive video processing unit. (1599)

My rule of thumb: The presence of a nested “if” tells you that you should draw the use case. (1605)

The Method prefers activity diagrams1 for graphical representation of use cases, primarily because activity diagrams can capture time-critical aspects of behavior, something that flowcharts and other diagrams are incapable of doing. (1611)

Software systems are typically designed in layers, and The Method relies heavily on layers. Layers allow you to layer encapsulation. Each layer encapsulates its own volatilities from the layers above and the volatilities in the layers below. (1626)

Even simple systems should be designed in layers to gain the benefit of encapsulation. In theory, the more layers, the better the encapsulation. Practical systems will have only a handful of layers, terminating with a layer of actual physical resources such as a data storage or a message queue. (1631)

The preferred way of crossing layers is by calling services. While you certainly can benefit from the structure of The Method and volatility-based decomposition even with regular classes, relying on services provides distinct advantages. Which technology and platform you use to implement your services is a secondary concern. (1634)

When you do use services (as long as the technology you chose allows), you immediately gain the following benefits: Scalability. (1636)

Security. All service-oriented platforms treat security as a first-class aspect. (1640)

Throughput and availability. Services can accept calls over queues, allowing you to handle a very large volume of messages by simply queuing up the excess load. (1642)

Responsiveness. Services can throttle the calls into a buffer to avoid maxing out the system. (1644)

Reliability. Clients and services can use some reliable messaging protocol to guarantee delivery, handle network connectivity issues, and even order the calls. (1645)

Consistency. The services can all participate in the same unit of work, either in a transaction (when supported by the infrastructure) or in a coordinated business transaction that is eventually consistent. (1646)

Synchronization. The calls to the service can be automatically synchronized even if the clients use multiple concurrent threads. (1649)

The Method calls for four layers in the system architecture. These layers conform to some classic software engineering practices. (1651)

The top layer in architecture is the client layer, also known as the presentation layer. I find the term “presentation” to be somewhat misleading. “Presentation” implies some information is being presented to human users, as if that is all that is expected from the top layer. The elements in the client layer may very well be end-user applications, but they can also be other systems interacting with your system. (1656)

The client layer also encapsulates the potential volatility in Clients. Your system now and in the future across the axes of volatility may have different Clients such as desktop applications, web portals, mobile apps, holograms and augmented reality, APIs, administration applications, and so on. (1667)

The business logic layer encapsulates the volatility in the system’s business logic. This layer implements the system’s required behavior, which, as mentioned previously, is best expressed in use cases. (1673)

Use cases, however, are volatile, across both customers and time. Since a use case contains a sequence of activities in the system, a particular use case can change in only two ways: Either the sequence itself changes or the activities within the use case change. (1675)

Both the sequence and the activities are volatile, and in The Method these volatilities are encapsulated in specific components called Managers and Engines. Manager components encapsulate the volatility in the sequence, whereas Engine components encapsulate the volatility in the activity. (1687)

Since use cases are often related, Managers tend to encapsulate a family of logically related use cases, such as those in a particular subsystem. (1693)

Since you can have great volatility in the sequence without any volatility in the activities of the sequence (see Figure 3-5), Managers may use zero or more Engines. Engines may be shared between Managers because you could perform an activity in one use case on behalf of one Manager and then perform the same activity for another Manager in a separate use case. (1698)

You should design Engines with reuse in mind. However, if two Managers use two different Engines to perform the same activity, you either have functional decomposition on your hands or you have missed some activity volatility. (1701)

The aptly named resource access layer encapsulates the volatility in accessing a resource, and the components in this layer are called ResourceAccess. (1704)

While the motivation behind the resource access layer is readily evident and many systems incorporate some form of an access layer, most such layers end up exposing the underlying volatility by creating a ResourceAccess contract that resembles I/O operations or that is CRUD-like. For example, if your ResourceAccess service contract contains operations such as Select(), Insert(), and Delete(), the underlying resource is most likely a database. If you later change the database to a distributed cloud-based hash table, that database-access-like contract will become useless, and a new contract is required. Changing the contract affects every Engine and Manager that has used the ResourceAccess component. (1710)

A well-designed ResourceAccess component exposes in its contract the atomic business verbs around a resource. (1721)

The Method refers to these indivisible activities as atomic business verbs. (1725)

Atomic business verbs are practically immutable because they relate strongly to the nature of the business, which, as discussed in Chapter 2, hardly ever changes. (1728)

ResourceAccess services can be shared between Managers and Engines. You should explicitly design ResourceAccess components with this reuse in mind. If two Managers or two Engines cannot use the same ResourceAccess service when accessing the same resource or have some need for specific access, perhaps you did not encapsulate some access volatility or did not isolate the atomic business verbs correctly. (1734)

The resource layer contains the actual physical Resources on which the system relies, such as a database, file system, a cache, or a message queue. In The Method, the Resource can be internal to the system or outside the system. Often, the Resource is a whole system in its own right, but to your system it appears as just a Resource. (1738)

The Method recommends the following conventions for naming them: Names of services must be two-part compound words written in Pascal case. The suffix of the name is always the service’s type—for example, Manager, Engine, or Access (for ResourceAccess). The prefix varies with the type of service. –  For Managers, the prefix should be a noun associated with the encapsulated volatility in the use cases. –  For Engines, the prefix should be a noun describing the encapsulated activity. –  For ResourceAccess, the prefix should be a noun associated with the Resource, such as data that the service provides to the consuming use cases. Gerunds (a gerund is a noun created by tacking “ing” onto a verb) should be used as a prefix only in with Engines. The use of gerunds elsewhere in the business or access layers usually signals functional decomposition. Atomic business verbs should not be used in a prefix for a service name. These verbs should be confined to operation names in contracts interfacing with the resource access layer. (1757)

The layers of services and resources in the architecture loosely correspond to the four English questions of “who,” “what,” “how,” and “where.” “Who” interacts with the system is in the Clients, “what” is required of the system is in Managers, “how” the system performs business activities is in Engines, “how” the system accesses Resources is in ResourceAccess, and “where” the system state is in Resources (see Figure 3-7). (1778)

If your design contains a large number of Engines, you may have inadvertently done a functional decomposition. (1803)

If your system has eight Managers, then you have already failed to produce a good design: The large number of Managers strongly indicates you have done a functional or domain decomposition. (1809)

In a well-designed system, volatility should decrease top-down across the layers. Clients are very volatile. (1821)

Managers do change, but not as much as their Clients. Managers change when the use cases—the required behavior of the system—change. (1824)

Engines are less volatile than Managers. For an Engine to change, your business must change the way it is performing some activity, which is more uncommon than changing the sequencing of activities. (1825)

ResourceAccess services are even less volatile than Engines. How often do you change the way you access a Resource or, for that matter, change the Resource? (1826)

Resources are the least volatile components, changing at a glacial pace compared with the rest of the system. (1829)

Reuse, unlike volatility, should increase going down the layers. (1833)

Clients are hardly ever reusable. A Client application is typically developed for a particular type of platform and market and cannot be reused. (1833)

Managers are reusable because you can use the same Manager and use cases from multiple Clients. (1835)

Engines are even more reusable than Managers because the same Engine could be called by multiple Managers, in different use cases, to perform the same activity. (1836)

ResourceAccess components are very reusable because they can be called by Engines and Managers. (1838)

The Resources are the most reusable element in any well-designed systems. (1838)

Managers can fall into one of three categories: expensive, expendable, and almost expendable. You can distinguish the category to which a Manager belongs by the way you respond when you are asked to change it. (1841)

If your response is to fight the change, to fear its cost, to argue against the change, and so forth, then the Manager was clearly expensive and not expendable. An expensive Manager indicates that the Manager is too big, likely due to functional decomposition. If your response to the change request is just to shrug it off, thinking little of it, the Manager is pass-through and expendable. (1843)

Expendable Managers are always a design flaw and a distortion of the architecture. They often exist only to satisfy the design guidelines without any real need for encapsulating use case volatility. (1846)

A well-designed Manager service should be almost expendable. (1851)

The Managers, Engines, and ResourceAccess are all services on their own right. A cohesive interaction between the Manager, Engines, and ResourceAccess may constitute a single logical service to external consumers. You can view such a set of interacting services as a logical subsystem. You group these together as a vertical slice of your system (Figure 3-8), where each vertical slice implements a corresponding set of use cases. (1853)

Avoid over-partitioning your system into subsystems. Most systems should have only a handful of subsystems. Likewise, you should limit the number of Managers per subsystem to three. (1858)

Design iteratively, build incrementally. (1870)

There are two reasons why you can build only incrementally, and not iteratively. First, building iteratively is horrendously wasteful and difficult (turning a motorcycle into a car is much more difficult than just building a car). Second, and much more importantly, the intermediate iterations do not have any business value. If the customer wants a car to take the kids to school, what would the customer do with a motorcycle and why should the customer pay for it? (1885)

The vertical slices of the system also enable you to accommodate extensibility. The correct way of extending any system is not by opening it up and hammering on existing components. (1898)

In common usage, microservices correspond to domains or subsystems—that is, to the slices (red boxes) of Figure 3-8. There are three problems with this idea as practiced today. (1923)

The first problem is the implied constraint on the number of services. If smaller services are better than larger services, why stop at the subsystem level? (1925)

The second problem is the widespread use of functional decomposition in microservice design by the industry at large. (1932)

The third problem relates to communication protocols. Although the choice of communication protocols has more to do with detailed design than with architecture, the effect of the choice is worth a passing comment here. (1937)

As a general principle, in any well-designed system you should never use the same communication mechanism both internally and externally. (1941)

Any layered architecture can have one of two possible operational models: open or closed. (1959)

In an open architecture, any component can call any other component regardless of the layer in which the components reside. (1962)

Calling sideways in this way is almost always the result of functional decomposition at the Managers level. (1981)

When using open architecture, there is hardly any benefit of having architectural layers in the first place. In general, in software engineering, trading encapsulation for flexibility is a bad trade. (1986)

In a closed architecture, you strive to maximize the benefits of the layers by disallowing calling up between layers and sideways within layers. (1988)

Closed architecture promotes decoupling by trading flexibility for encapsulation. In general, that is a better trade than the other way around. (1992)

A semi-closed/semi-open architecture allows calling more than one layer down. This, again, is a trade of encapsulation for flexibility and performance and, in general, is a trade to avoid. (1997)

Notably, the use of semi-closed/semi-open architecture is justified in two classic cases. This first case occurs when you design some key piece of infrastructure, and you must squeeze every ounce of performance from it. (1999)

The second case occurs within a codebase that hardly ever changes. (2004)

In a closed architecture, Utilities pose a challenge. Consider Logging, a service used for recording run-time events. If you classify Logging as a Resource, then the ResourceAccess can use it, but the Managers cannot. If you place Logging at the same level as the Managers, only the Clients can log. The same goes for Security or Diagnostics—services that almost all other components require. In short, there is no good location for Utilities among the layers of a closed architecture. The Method places Utilities in a vertical bar on the side of the layers (see Figure 3-4). This bar cuts across all layers, allowing any component in the architecture to use any Utility. (2016)

To qualify as a Utility, the component must pass a simple litmus test: Can the component plausibly be used in any other system, such as a smart cappuccino machine? (2026)

This next guideline may be implied, but it is important enough to state explicitly. Because they are in the same layer, both Managers and Engines can call ResourceAccess services without violating the closed architecture (see Figure 3-4). Allowing Managers to call ResourceAccess is also implied from the section defining Managers and Engines. A Manager that uses no Engines must be able to access the underlying Resources. (2033)

Managers can directly call Engines. The separation between Managers and Engines is almost at the detailed design level. Engines are really just an expression of the Strategy design pattern6 used to implement the activities within the Managers’ workflows. Therefore, Manager-to-Engine calls are not truly sideways calls, as is the case with Manager-to-Manager calls. (2038)

While Managers should not call directly sideways to other Managers, a Manager can queue a call to another Manager. (2046)

Using The Method structure, when a Manager queues a call to another Manager, the proxy is a ResourceAccess to the underlying Resource, the queue; that is, the call actually goes down, not sideways. (2051)

The semantic explanation involves the nature of use cases. Business systems quite commonly have one use case that triggers a latent, much-deferred execution of another use case. For example, imagine a system in which a Manager executing a use case must save some system state for analysis at the end of the month. Without interrupting its flow, the Manager could queue the analysis request to another Manager. The second Manager could dequeue at the month’s end and perform its analysis workflow. The two use cases are independent and decoupled on the timeline. (2054)

If you do one of the things on this list, you will likely live to regret it. Treat any violation of these rules as a red flag and investigate further to see what you are missing: (2078)

Another universal design rule is that all good architectures are symmetric. (2119)

The quest for symmetry, however, is only at the architecture level, not in detailed design. (2121)

The symmetry in software systems manifests in repeated call patterns across use cases. You should expect symmetry, and its absence is a cause for concern. (2124)

For example, suppose a Manager implements four use cases, three of which publish an event with the Pub/Sub service and the fourth of which does not. That break of symmetry is a design smell. Why is the fourth case different? What are you missing or overdoing? Is that Manager a real Manager, or is it a functionally decomposed component without volatility? Symmetry can also be broken by the presence of something, not just by its absence. (2125)

But how do you know the composition of these components at run-time adequately satisfies all the requirements? (2135)

Design validation and composition, as you will see, are intimately related. (2137)

Never design against the requirements. (2157)

As discussed in Chapter 3, the correct way of capturing the requirements is in the form of use cases: the required set of behaviors of the system. (2166)

The core use cases represent the essence of the business of the system. As discussed in Chapter 2, the nature of the business hardly ever changes, and the same goes for the core use cases. (2187)

Most systems have as few as two or three core use cases, and the number seldom exceeds six. (2193)

A core use case will almost always be some kind of an abstraction of other use cases, and it may even require a new term or name to differentiate it from the rest. (2199)

The whole point of requirements analysis is to recognize the core use cases (and the areas of volatility). (2202)

Your mission as an architect is to identify the smallest set of components that you can put together to satisfy all the core use cases. (2205)

I call this approach composable design. A composable design does not aim to satisfy any use case in particular. You do not target any use case in particular not just because the use cases you were given were incomplete, faulty, and full of holes and contradictions, but because they will change. (2214)

Since the goal of any system is to satisfy the requirements, composable design enables something else: design validation. Once you can produce an interaction between your services for each core use case, you have produced a valid design. (2222)

Figure 4-1 is, in The Method’s parlance, a call chain diagram. (2230)

A call chain demonstrates the interaction between components required to satisfy a particular use case. You can literally superimpose the call chain onto the layered architecture diagram. (2233)

The components in the diagram are connected by arrows indicating the direction and type of the call between components—a solid black arrow for synchronous (request/response) calls, and a dashed gray arrow for a queued call. (2234)

A sequence diagram in The Method’s parlance is similar to a UML sequence diagram.1 However, it includes notational differences to assure common meanings between diagram types. (2242)

Remember: Your mission as the architect is to identify not just a set of components that you can put together to satisfy all the core use case, but the smallest set of components. (2258)

The smallest set of services required in a typical software system contains 10 services in order of magnitude (e.g., sets of both 12 and 20 are on the order of 10). This particular order of magnitude is another universal design concept. (2270)

Using The Method, even in a large system you are commonly looking at two to five Managers, two to three Engines, three to eight ResourceAccess and Resources, and a half-dozen Utilities. (2276)

You may spend weeks or months trying to identify the core use cases and the areas of volatility. However, that is not design—that is requirements gathering and requirements analysis, which may be very time-consuming indeed. (2286)

Features are always and everywhere aspects of integration, not implementation. (2293)

The first order of business is to get all stakeholders to agree on a common vision. The vision must drive everything, from architecture to commitments. Everything that you do later has to serve that vision and be justified by it. Of course, this cuts both ways—which is why it is a good idea to start with the vision. If something does not serve the vision, then it often has to do with politics and other secondary or tertiary concerns. This provides you with an excellent way of repelling irrelevant demands that do not support the agreed-upon vision. (2556)

A platform for building applications to support the TradeMe marketplace. (2561)

Vision → Objectives → Mission Statement → Architecture (2593)

This is a reversal of the typical dynamics, in which the architect pleads with management to avoid functional decomposition. It is a lot easier to drive the correct architecture through the business by aligning the architecture with the business’s vision, its objectives, and the mission statement. (2595)

Before you dive into the act of system design, ensure everyone is on the same page by compiling a short glossary of domain terminology. (2602)

A good way of starting a glossary is to answer the four classic questions of “who,” “what,” “how,” and “where.” You answer the questions by examining the system overview, the use cases, and customer interview notes, if you have any. (2604)

For TradeMe, the answers to the four questions were as follows: (2606)

This does not preclude having additional subsystems or imply that these will necessarily be all the subsystems needed—you always decompose based on volatility, and if a “what” is not volatile, then it will not merit a component in the architecture. (2619)

There is nothing wrong with suggesting certain areas of volatility, and then examining the resultant architecture. If the result produces a spiderweb of interactions or is asymmetric, then the design is unlikely to be good. (2641)

The main reason for choosing a message bus is because it supports the most important operational concept of TradeMe: the Message Is the Application design pattern. When using this design pattern, the “application” is nowhere to be found. There is no collection of components or services that you can point to and identify as the application. Instead, the system comprises a loose collection of services that post and receive messages to one another (over a message bus, although that is secondary consideration). These messages are related to each other. (2777)

The use of granular services integrated over a message bus with the Message Is the Application design pattern is one of the best ways of preparing the system for the future. By “preparing the system for the future,” I specifically refer to the next epoch in software engineering, the use of the actor model. (2796)

For more on the actor model, see Juval Lowy, Actors: The Past and Future of Software Engineering (YouTube/IDesignIncTV, 2017). (2809)

In many cases, a simpler design in which the Clients just queue up calls to the Managers would be a better fit for the development team. Always calibrate the architecture to the capability and maturity of the developers and management. After all, it is a lot easier to morph the architecture than it is to bend the organization. Once the organizational capabilities have matured, you can incorporate a full Message Is the Application pattern. (2815)

A workflow Manager is a service that enables you to create, store, retrieve, and execute workflows. In theory, it is just another Manager. In practice, however, such Managers almost always utilize some sort of third-party workflow execution tool and workflow storage. For each Client call, the workflow Manager loads not just the correct workflow type but also a specific instance of it, with a particular state and context; executes the workflow; and persists it back to the workflow store. (2824)

6. Motivation (3006)

The project needs can be classified into five levels: physical, safety, repeatability, engineering, and technology. (3065)

Physical. This is the lowest level in the project pyramid of needs, dealing with physical survival. Much as a person must have air, food, water, clothing, and shelter, a project must have a workplace (even a virtual one) and a viable business model. (3066)

Safety. Once the physical needs are satisfied, the project must have adequate funding (often in the form of allocated resources) and enough time to complete the work. (3070)

Repeatability. Project repeatability describes the development organization’s ability to deliver successfully time and again, and is the foundation for control and execution. It assures that if you plan for and commit to a certain schedule and cost, you will deliver on those commitments. (3074)

Engineering. Once the repeatability of the project effort is secured, the software project can, for the first time, turn its attention to the enticing aspects of software engineering. This includes architecture and detail design, quality assurance activities such as root cause analysis and correction (on a systemic level), and preventive work using hardened operating procedures. (3078)

Technology. At this level are development technology, tools, methodology, operating systems, and related hard-core technical aspects. (3082)

Part 1 of this book stated a universal design rule: Features are always and everywhere aspects of integration, not implementation. As such, there are no features in any of the early services. At some point you will have integrated enough to start seeing features. I call that point the system. (3121)

Never base progress reports on features. Always base progress reports on integration. (3137)

A single architect is absolutely crucial for design integrity. You can extend this observation to the general rule that the only way to allow for design integrity is to have a single architect own the design. The opposite is also true: If no single person owns the design and can visualize it cover-to-cover, the system will not have design integrity. (3168)

As vital as the architect is to the project, the architect cannot work in isolation. On day 1, the project must have a core team in place. The core team consists of three roles: project manager, product manager, and architect. (3191)

The project manager. The job of the project manager is to shield the team from the organization. Most organizations, even small ones, create too much noise. If that noise makes its way into the development team, it can paralyze the team. A good project manager is like a firewall, blocking the noise, allowing only sanctioned communication through. (3195)

The product manager. The product manager should encapsulate the customers. Customers are also a constant source of noise. The product manager acts as a proxy for the customers. For example, when the architect needs to clarify the required behaviors, the architect should not chase customers; instead, the product manager should provide the answers. (3201)

The architect. The architect is the technical manager, acting as the design lead, the process lead, and the technical lead of the project. The architect not only designs the system, but also sees it through development. (3205)

The core team designs the project in the fuzzy front end leading to development. The fuzzy front end is a general term1 in all technical projects referring to the very start of the project. The front end commences when someone has an idea about the project, and it concludes when developers start construction. (3225)

The result of project design is a set of plans, not a single plan. (3255)

Officially, the name of the meeting should be the Software Development Plan Review, or SDP review. It makes no difference if your process does not have an SDP review point: Just call a meeting (no manager can refuse a meeting request whose subject line is “Software Development Plan Review”). Once the desired option is identified, management must literally sign off on the SDP document. This document now becomes your project’s life insurance policy because, as long as you do not deviate from the plan’s parameters, there is no reason to cancel your project. This does require proper tracking (as described in Appendix A) and project management. (3269)

For now, recognize that you should always assign services to developers in a 1:1 ratio. The 1:1 ratio does not mean that a developer works on only one service, but rather that if you do a cross-section of the team at any moment in time, you will see a developer working on one and only one service. (3284)

Any other way of assigning services to developers will result in failure. Examples of the poor assignment options include: Multiple developers per service. The motivation for assigning two (or more) developers to one service is not a surplus of developers, but rather the desire to complete the work sooner. However, two people cannot really work on the same thing at the same time, so some subscheme must be used: (3288)

Multiple services per developer. The option of assigning two (or more) services to a single developer is just as bad. Suppose two services, A and B, each estimated as a month of work, are assigned to a single developer, with the developer expected to finish both after a single month. Since the sum of work is two months, not only will the services be incomplete after one month, but finishing them will take much longer. While the developer is working on the A service, the developer is not working on the B service, causing the developers dependent on the B service to demand that the developer work on the B service. (3302)

When assigning services (or activities such as UI development), try to maintain task continuity, a logical continuation between tasks assigned to each person. Often, such task assignments follow the service dependency graph. If service A depends on service B, then assign A to the developer of B. One advantage is that the A developer who is already familiar with B needs less ramp-up time. (3353)

Effort estimation is how you try to answer the question of how long something will take. There are two types of estimations: individual activity estimation (estimating the effort for an activity assigned to a resource) and overall project estimation. The two types of estimations are unrelated, because the overall duration of the project is not the sum of effort across all activities divided by number of resources. (3366)

Overestimation never works because of Parkinson’s law. (3382)

Good estimations are accurate, but not precise. For example, consider an activity that actually took 13 days and had 2 estimations: 10 days or 23.8 days. While the second estimation is far more precise, clearly the first estimation is better because it is more accurate. (3424)

Estimations also must match the tracking resolution. If the project manager tracks the project on a weekly basis, any estimation less than a week is pointless because it is smaller than the measurement resolution. (3429)

Confronted with the uncertain, take these steps: Ask first for the order of magnitude: Is the activity more like a day, a week, a month, or a year? (3444)

Make an explicit effort to list the areas of uncertainty in the project and focus on estimating them. Always break down large activities into smaller, more manageable activities to greatly increase the accuracy of the estimations. (3449)

Invest in an exploratory discovery effort that will give insight into the nature of the problem and reduce the uncertainty. (3450)

One estimation technique dealing specifically with high uncertainty is part of Program Evaluation and Review Technique (PERT).3 For every activity, you provide three estimations: the most optimistic, the most pessimistic, and the most likely. (3453)

With even a modest degree of repeatability (see Figure 6-1), it is unlikely that you could deliver the project faster or slower than similar projects in the organization’s past. The dominant factor in throughput and efficiency is the organization’s nature, its own unique fingerprint of maturity, which is something that does not change overnight or between projects. If it took your company a year to deliver a similar project in the past, then it will take it a year in the future. (3474)

Some tools even use Monte Carlo simulations to narrow down the range of the variables based on your project attributes or historical records. I have used such tools for decades, and they produce accurate results. (3483)

The broadband estimation is my adaptation of the Wideband Delphi4 estimation technique. The broadband estimation uses multiple individual estimations to identify the average of the overall project estimation, then adds a band of estimations above and below it. You use the estimations outside the band to gain insight into the nature of the project and refine the estimations, repeating this process until the band and the project estimations converge. 4. Barry Boehm, Software Engineering Economics (Prentice Hall, 1981). (3485)

You start the project design with the estimated duration of the individual activities in the project. Before you estimate individual activities, you must prepare a meticulous list of all activities in the project, both coding and noncoding activities alike. In a way, even that list of activities is an estimation of the actual set of activities, so the same rationale about reducing uncertainties holds true here. (3518)

If you ask others to estimate an activity, you must maintain a correct estimation dialog with them. Never dictate duration by saying, “You have two weeks!” Not only is that based on nothing, but the owner of the activity also does not feel accountable to actually finish in two weeks. When people are unaccountable, progress and quality will be lacking. Avoid leading questions, such as “It is going to take two weeks, right?” While this is somewhat better than dictating the estimation, you now bias the other party toward your estimation. (3533)

Critical path analysis is the single most important project design technique. However, you cannot perform this analysis without the following prerequisites: The system architecture. You must have the decomposition of the system into services and other building blocks such as Clients and Managers (3542)

A list of all project activities. Your list must contain both coding and noncoding activities. (3547)

Activity effort estimation. Have an accurate estimation of the effort for each activity in the list of activities. (3550)

Services dependency tree. Use the call chains to identify the dependencies between the various services in the architecture. (3551)

Activity dependencies. Beyond the dependencies between your services, you must compile a list of how all activities depend on other activities, coding and noncoding alike. (3552)

Planning assumptions. You must know the resources available for the project or, more correctly, the staffing scenarios that your plan calls for. (3554)

You can graphically arrange the activities in the project into a network diagram. The network diagram shows all activities in the project and their dependencies. You first derive the activity dependencies from the way the call chains propagate through the system. (3558)

You should turn the diagram in Figure 7-4 into the detailed abstract chart shown in Figure 7-5. That chart now contains all activities, coding and noncoding alike, such as architecture and system testing. (3573)

The time to get to an activity, or the time it takes to be ready to start working on the activity, is the maximum of time of all network paths leading to that activity. In a more formal manner, you calculate the time for completing activity i in the project with this recursive formula: where: Ti is the time for completing activity i. Ei is the effort estimation for activity i. n is the number of activities leading directly to activity i. (3580)

By calculating the activity times, you can identify the longest possible path in the network of activities. In this context, the longest path means the path with greatest duration, not necessarily the one with the greatest number of activities. (3606)

Based on the effort estimation for each activity and the dependencies, using the formula given earlier and starting from activity 17, the longest path in the network is shown in bold. That longest path in the network is called the critical path. (3611)

With so few developers, you will paint yourself into a corner in which a developer on the critical path needs a noncritical activity that is simply not ready yet (such as activity 15 needing activity 11). This promotes a noncritical activity to a critical activity, in effect creating a new and longer critical path. I call this situation subcritical staffing. When the project goes subcritical, it will miss its deadline because the old critical path no longer applies. (3659)

All noncritical activities have float, which is the amount of time you could delay completing them without delaying the project. Critical activities have no float (or more precisely, their float is zero) since any delay in these activities would delay the project. When you assign resources to the project, follow this rule: Always assign resources based on float. (3691)

Together, the project network, the critical path, and the float analysis allow you to calculate the duration of the project as well as when each activity should start with respect to the project beginning. (3760)

If you have several projects in the organization, then you could arrange them such that developers are always phasing out of one project while phasing into another. Working this way yields a hundreds of percent increase in productivity, the classic “doing much more with less.” (3780)

You produce a chart such as Figure 7-9 by first staffing the project, then listing all the dates of interest (unique dates when activities start and end) in chronological order. You then count how many resources are required for each category of resources in each time period between dates of interest. (3797)

The efficiency of a project is the ratio between the sum of effort across all activities (assuming perfect utilization of people) and the actual project cost. (3889)

The expected efficiency of a well-designed system, along with a properly designed and staffed project, ranges between 15% and 25%. (3893)

The earned value at time t is the ratio between the sum of estimated duration of all activities completed by time t divided by the sum of the estimated durations of all activities. (3930)

The earned value curve is a simple and easy way to answer the question: “Does the plan make sense?” If the planned earned value is a straight line, or it exhibits the issues of Figure 7-20 or Figure 7-22, the project is in danger. If it looks like a shallow S, then at least you have hope that the plan is sound and sensible. (4014)

The project network acts as a logical representation of the project for planning purposes. The technique for analyzing the network is called the critical path method, although it has as much to do with the noncritical activities as it does with the critical ones. (4038)

An activity in a software project is any task that requires both time and a resource. (4047)

The project is a collection of related activities, and the network diagram captures these activities and their dependencies. In network diagrams, there is no notion of order of execution or concurrency between the activities. (4048)

Consequently, you should avoid node diagrams and use arrow diagrams. The initial arrow diagram learning curve is more than offset by the benefits of having a concise, clear, clutter-free model of your project. (4116)

An activity’s total float is by how much time you can delay the completion of that activity without delaying the project as a whole. (4151)

An activity’s free float is by how much time you can delay the completion of that activity without disturbing any other activity in the project. (4169)

As stated in Chapter 7, the safest and most efficient way to assign resources to activities is based on float—or, given the definition of this chapter, total float. This is the safest method because you address the riskier activities first, and it is the most efficient because you maximize the percentage of time for which the resources are utilized. (4243)

As just stated, assigning resources based on float allows you to trade float for cost. You may be tempted to trade all the project’s float for lower cost, but that is rarely a good idea because a project with little float has less tolerance for delays. (4289)

In general, the following techniques are possible in any software project and will accelerate the project as a whole: Assure quality. (4306)

Employ test engineers. (4314)

Add software testers. (4322)

Invest in infrastructure. (4327)

Improve development skills. (4332)

Improve the process. (4339)

Adopt and employ standards. (4346)

Provide access to external experts. (4350)

Engage in peer reviews. (4355)

However, you can do two things to immediately accelerate the schedule—either work with better resources or find ways of working in parallel. (4370)

Schedule compression means accomplishing the same objectives faster, often by doing more work to finish the task or the project sooner. You can use these two compression techniques in combination with each other or in isolation, on parts of the project, on the project as a whole, or on individual activities. Both compression techniques end up increasing the direct cost (defined later) of the project while reducing the schedule. (4372)

Removing dependencies often involves investing in additional activities that enable the parallel work in the first place: Contract design. (4405)

Emulators development. (4409)

Simulators development. (4411)

Repeated integration and testing. (4414)

In Perspective (7204)