Distributed Systems

This article aims to provide an accessible yet thorough introduction to distributed systems in 2023, which have evolved into a crucial component of contemporary data processing systems. This manual is intended for those who want to work in distributed systems or are beginning their careers in this field. For seasoned experts who need a refresher, the guide may be helpful.

What do we mean by a distributed system?

It is a distributed system, as the name would suggest. This system's parts function as a single, well-integrated unit. In comparison to a non-distributed system, it is also far more fault resistant and easily horizontally scaleable. It commonly referred to as distributed computing, is a set of interconnected components running on many computers that work together to communicate and coordinate tasks so that the end user sees just one cohesive system. Computers, physical servers, virtual machines, containers, and other nodes with network connectivity, local memory, and message passing capabilities can all be considered to be part of a distributed system.

In general, distributed systems operate in one of two ways:
The end user sees the results as a coherent whole since each machine is working toward a single objective. Since each machine has a unique end user, sharing resources or communication services is made possible by the distributed system.

All components run simultaneously, there is no global clock, and all components fail independently of one another, despite the fact that distributed systems can occasionally be obscure.

Why is a distributed system necessary?

In general, they enable us to accomplish the following
Horizontal scaleability
high efficiency given the infrastructure expenses
a lot of availability

Concurrent and Distributed

Almost always, concurrent code is operating on a single machine as part of a distributed system. Typically, you have to execute timers, separate business logic, network interfaces, and so forth. things where concurrency is frequently a useful tool.

In addition to being intrinsically concurrent, a distributed system is also one in which the concurrent units (which contain further concurrent sub-units; see 1), can dissipate, lose state, and reappear at any time.A distributed programme would add a third dimension if it were distributed instead of being sequential, making concurrent and distributed programmes two dimensions instead of one.

Benefits and challenges of distributed systems

Teams typically decide to implement distributed systems for one of three reasons:
Horizontal Scalability: Adding extra nodes and functionality as needed is simple and often affordable because computing occurs independently on each node.
Reliability—Most distributed systems can have hundreds of cooperating nodes, making them fault-tolerant in most cases. If one machine breaks, the system often doesn't encounter any interruptions.
Performance—Because work loads may be divided up and distributed among several machines, distributed systems are incredibly effective.

Distributed systems do have some difficulties, though. It can be overwhelming to go through the intricate architectural design, development, and debugging processes needed to build a successful distributed system.

There are still three obstacles you might face.
A distributed system must decide which tasks must be executed, when they should be executed, and where they should be executed. Ultimately, schedulers have their limits, which results in underutilised hardware and unpredictable runtimes.
Lag—The more globally distributed your system is, the more communications latency you may encounter. Teams frequently compromise between availability, consistency, and latency as a result of this.
Observability—For big clusters, collecting, analysing, presenting, and keeping track of hardware consumption metrics is a difficult task.

Types of Distributed Systems

A distributed system is a network of devices that may communicate with one another by delivering messages. As it promotes resource sharing, it can be quite beneficial.

Client/Server Systems: The client asks resources or a task be completed from the server, which then assigns the requested resource or completes the task and provides the completed task or result back to the client.

Peer to Peer Systems: Nodes play a key role in a system. Each node in this can function as a server or a client for a system, doing their own tasks on their local memory and sharing data across the supporting channel.

Middleware serves as a foundation for various interoperability programmes that operate on various operating systems. By utilising this service, data can be transmitted between parties.

Three-tier: This method allows for easy development because client data is held in the middle tier rather than being sorted into the client system or on their server. Most web or internet apps use this.
N-tier: When interoperability asks another application to carry out a function or offer a service.

How distributed system work?

Although distributed systems have changed over time, the majority of today's implementations are still built to work online, specifically in the cloud. A distributed system starts with a task, such generating a movie to produce an output that is ready for distribution.

The job is divided into portions by the online application, or distributed apps, handling it, similar to a client computer's video editor. In this straightforward illustration, the algorithm assigns one video frame to each of a dozen distinct computers (or nodes) so that they can produce the entire video. The management application provides the node with a new frame to operate on after the previous frame has been finished. As soon as all the pieces have been placed back together, the video is complete.

The job can be split among hundreds or even thousands of nodes in a system like this, making it possible to perform a task that would have taken a single computer days to complete in a matter of minutes. This system doesn't have to stop at just 12 nodes.

Distributed Systems Use Cases

There are various justifications for designing a distributed system. For instance, large-scale matrix multiplications are required when creating machine learning models. These cannot be supported by a single machine. Similar to this, systems that handle enormous files and process and store them on a single machine may simply be impossible or at the very least extremely inefficient.

Therefore, we can generally group distributed systems into the following groups depending on the use-case. However, this is by no means a complete list of distributed systems' potential use cases.

Datastores
Messaging
Computing
Ledgers
File-systems
Applications

Relational databases have historically been the go-to option for data storage for a long time. Relational databases, however, started to fall short of expectations as data volume, diversity, and velocity increased recently. It was at this point that NoSQL databases' distributed architecture began to show more promise.

Similarly, the limitations of the current data size prevented traditional communications systems from remaining impervious to them. As a result, the demand for distributed messaging systems with performance, scalability, and perhaps durability began to grow. Presently, a number of choices in this field offer multiple semantics, including publish-subscribe and point-to-point.

In this tutorial, we'll talk about some well-known distributed databases and messaging systems. The general architecture and how it handles some of the major difficulties of distributed systems, such as partitioning and coordination, will be the main areas of focus.

Apache Cassandra

A distributed key-value system that uses an open-source partitioned wide column storage model is called Cassandra. It has comprehensive multi-master data replication, which offers minimal latency and high availability. It has no single point of failure and is linearly scalable.

Cassandra is an eventually consistent database because it prioritises high availability and scalability. In essence, this means that all data updates finally reach every copy. Divergent iterations of the same data, however, may exist momentarily. For both read and write operations, Cassandra also offers configurable consistency in the form of a list of available consistency levels.

Mongo DB

An open-source, document-based, distributed database called MongoDB stores data as a set of documents. A document is a straightforward data structure made up of pairs of fields and values. Additionally, it offers embedded documents and arrays for sophisticated data modelling.

Shards of MongoDB can be put up as replica sets. A replica set's principal member responds to all queries. During an automated failover, the shard is typically still unavailable to handle queries. Due to this, MongoDB is very consistent by design. However, a client can select to read from a secondary replica where the data is only sporadically consistent in order to increase high availability.

Apache Kafka

A unified, high-throughput, low-latency solution for handling real-time data flows is provided by the open-source Kafka platform. We can store streams of events durably and reliably, broadcast and subscribe to streams of events, and process streams of events either in real time or after the fact.
Kafka replicates data over several nodes with automatic failover to improve durability and availability. Only after all in-sync replicas have digested an event is it deemed committed. Additionally, only loyal customers can get messages. Because to this, Kafka has a wide range of options and is made to be very consistent and available.

Decentralized vs distributed

Technically speaking, decentralised still refers to distributed networks, but the entire decentralised system is not controlled by a single player. A decentralised system cannot be owned by one firm; otherwise, it would cease to be decentralised.

As a result, the majority of the systems we'll discuss today may be categorised as distributed centralised systems because that's what they were designed to be.

If you give it some thought, building a decentralised system is more difficult since you have to deal with members who are nasty. With typical distributed systems, this is not the case because you are aware that every node belongs to you.

In this article, we went through the fundamentals of a distributed system and identified its main advantages and disadvantages.

Good luck and happy learning!