It all starts with bits: digital units of information. It is about how we store these bits, how we move these bits, and how we transform or process these bits. The three major elements of digital infrastructures are: Storage Networks Processors In the cloud security world, it is customary to talk about data at rest, data in motion, and data in use. This introduces data as the key common element. But more on cloud security later. ...
You should not read this book because I tell you to. After all, you are an autonomous person, making your own decisions. You should read this book because you are convinced that it is likely to give you significant insights into a world that matters to you. Digital infrastructures, and information technology at large, have moved from a fringe niche technology to becoming an essential part of modern life. Most companies would go belly up in weeks, if not days, if their digital infrastructures would fail. ...
Digital infrastructures focus on services rather than products, making it elusive to capture the essence of these services, especially in defining the fluid interactions between service providers and consumers. Promise theory is a little-known approach to interaction, though it has quite a few great thinkers behind it. Introduced by Mark Burgess and others, it is about how autonomous agents work together. It helped me a lot in getting a handle on various distributed systems. There was this project where we designed a scalable travel information system with what we now call microservices. We needed to describe how these parts work together. And there was a project that required lots of independent and autonomous social security agencies to work together on providing a service. In getting this from the drawing board to actual production, promise theory turned out to be a great tool. ...
Contracts are complementary promises Once we understand promises, contracts between agents now become really simple to express. They are a set of complementary conditional promises: “If you do this, I will promise that”. In our example this looks like the following. “If you promise to pay me, I will promise to bring you coffee.” “If you promise to bring me coffee, I will promise to pay you.” If one party does not keep its end of the bargain, the other party is free to withdraw from their part of the deal. Of course there is the complication that somebody has to deliver first: do you get coffee first, or do you have to pay first? This requires trust. As you are probably aware, both types of contract exist in real life. In fact you can find more complicated versions as well. ...
Computers are terribly reliable, in general. Today’s computer systems execute millions, even trillions, of instructions each second, with an error rate that is inconceivable in other technologies. Yet, if you have hundreds of thousands of machines, you do need to take care of failures. In the early days of Google growth I read an article about their error numbers (a Google cluster has several thousands of machines): In each cluster’s first year, it’s typical that 1,000 individual machine failures will occur; thousands of hard drive failures will occur; one power distribution unit will fail, bringing down 500 to 1,000 machines for about 6 hours; 20 racks will fail, each time causing 40 to 80 machines to vanish from the network; 5 racks will “go wonky,” with half their network packets missing in action; and the cluster will have to be rewired once, affecting 5 percent of the machines at any given moment over a 2-day span, Dean said. And there’s about a 50 percent chance that the cluster will overheat, taking down most of the servers in less than 5 minutes and taking 1 to 2 days to recover. ...
What do roads, airports, the internet, the electricity network, and a search engine have in common? They are all services that are independent of a specific user or usage. They are provisioned on a longer timescale than that of an individual usage. And they are typically not owned by their users, or at least, not directly. In this book I am mostly concerned with digital infrastructures. Their services are digitally accessible. Of the above examples, the internet and the search engine are the best examples of that. Interestingly, the other services increasingly rely on digital infrastructures themselves, or even incorporate specific digital infrastructures. An example of that is the trading platforms that enable the planning of electricity supply and demand. In many countries, electricity is a market, not a monopoly, which requires coordination between the various players. ...
In 2005 I visited my father in France. It turned out to be convenient to pick me up from the Gerona airport in Spain. And as we had enough time, we had lunch in La Jonquera. At the restaurant’s checkout I noticed a peculiar array of devices: there were four payment terminals. I’d never seen that. From what I know, a merchant works with a bank which handles all their payments. Apparently not over here. ...
What do a laptop, a smartphone, and a smart thermostat have in common with a browser, a database, and a data center? They are all execution environments that contain software and data, and that makes them building blocks for deploying digital infrastructures. In a diagram we often depict them as a box, or an oval. Inside the box, software gets executed, instructions get interpreted, actions are done. Software without execution is just dead data. ...
Deployment is everything that happens between writing software and actually using that software by its intended users. And as we get more software and more users, deployment becomes more complex. Why deployment diagrams? Deployment diagrams are a great technique for communicating about important decisions in deploying software. Decisions such as who is going to do what, how are things connected, and so on. There are many ways to draw deployment diagrams and many standards to choose from. UML and Archimate are just a few of them. To me, there is no single right way to create deployment diagrams. In that sense, these diagrams are more like maps. And the usefulness of a map depends on the journey that you are going to make. A map for a mountain walk is pretty useless if you want to make a railroad journey and vice versa. ...
The AI landscape has many digital infrastructures. Let’s explain this step by step and focus on which data is stored where, and how it is processed. A core element of AI systems is a trained model. At least that is true for the dominant AI form these days: deep learning neural networks. A trained model is the result of processing a lot of training data by a specific neural network. These models are fixed in size, but typically very big. The smallest useful models are close to a Gigabyte, while recent public chat models run into multiple Terabytes. ...