What is concurrent programming and how to master it

Concurrent programming has become a key component of modern software development: operating systems, web servers, mobile apps, video games, databases… almost everything we use daily executes multiple tasks simultaneously, or at least appears to. A solid understanding of how this programming approach works, how it differs from parallelism, and the challenges it presents is essential for writing robust and efficient code.

In the following lines, we'll break down exactly what concurrency is, how it's modeled at the process and thread level, what models exist (threads, coroutines, actors, event loops, etc.) , what typical errors arise (race conditions, deadlocks, starvation, false sharing, etc.), and what tools we have for synchronizing and verifying concurrent systems. You'll see that, beyond the theory, all of this has a very practical application in real-world software design.

What is concurrent programming and how does it differ from parallelism?

When we talk about concurrency, we're referring to a system's ability to manage multiple tasks whose lifecycles overlap , so their execution steps intertwine. The key concept is logical overlap: from the programmer's perspective, several activities are "live" at the same time, even though physically a single processor can only execute one instruction at a time.

Execution is organized as an interleaving process : the scheduler allocates small portions of CPU time to each task. It executes a portion of task A, then moves on to task B, then to task C, and later returns to task A, always respecting data dependencies. Concurrent programming, therefore, is a way of structuring programs into several nearly independent activities , so that the exact order in which they are interleaved is not strictly fixed.

Parallelism is another story: here we're talking about several tasks running literally at the same time on different cores or processors ( managing the CPU cores ). A system can be concurrent but not parallel (a single core switching tasks) or both concurrent and parallel (multiple cores executing some of these concurrent threads or processes in parallel).

In practice, concurrency aims to model problems where multiple things happen simultaneously (web requests, user clicks, disk or network I/O, sensors, etc.), while parallelism focuses on accelerating heavy computations by dividing them into subproblems that run concurrently. A single concurrent design can run in a merely interleaved manner or exploit true parallelism if the hardware has multiple cores.

Processes, threads, and execution states

In an operating system, the basic concept for supporting concurrency is the process . A process is not just the executable file on disk: it is an active entity with its own execution state. It includes a program counter (a CPU register that points to the next instruction to be executed), a process stack (function parameters, return addresses, local variables, etc.), a data section (global variables, static data), and a heap where dynamic memory is allocated at runtime.

The binary file stored on disk is merely a passive program , a sequence of instructions and data. For that program to "come to life," the operating system must create a process, initialize its context (registers, memory, file descriptors, etc.), and schedule it to receive CPU time.

States of a process

While a process exists, it goes through different execution states , which are usually at least the following:

• NewThe system is creating the process and configuring its internal structures.
• RunningThe process has the CPU and its instructions are being carried out.
• Blocked or on hold: the process waits for an event to occur (input/output, end of an operation, arrival of a message…).
• Ready or preparedIt has everything it needs to run, but it's waiting its turn on the CPU.
• FinishedIts execution has finished and the system will release its resources.

The switching between these states is decided by the operating system scheduler according to its policy (round robin, priorities, etc.), balancing CPU usage, latency, and fairness between processes.

Process control block (PCB)

Each process is represented in the kernel by a data structure known as a PCB (Process Control Block) . This block is the "file" of the process and stores all the information necessary to stop and resume it without losing anything.

The PCB stores the process state (new, ready, running, blocked, etc.), the current program counter , CPU registers (whose number and type depend on the architecture), scheduling information (priority, pointers to ready queues, usage statistics), and memory management data (page tables, segment tables, etc.). This allows the operating system to perform context switches , that is, to save the complete state of one process and restore the state of another.

Threads and multi-threads

In addition to processes, many operating systems provide threads of execution . A thread is a basic unit of CPU usage: it has its own identifier, program counter, registers, and stack, but it shares the code section, data section, and other resources such as file descriptors with other threads of the same process.

In a multithreaded program, multiple threads can execute different parts of the same code in parallel or interleaved . Each maintains its own stack (with its local variables and call context), but all see the same process address space. This facilitates communication (by sharing memory) at the cost of introducing concurrency risks such as race conditions.

In a typical multithreaded design, the process has a single PCB and a single address space, while each thread adds its own stack and register context . The operating system (or language runtime) implements a preemptive model: it can interrupt a thread at any time and give the CPU to another, forcing you to program with the understanding that your thread can be paused between two seemingly "atomic" instructions.

Multiprogramming, multiprocessing, and distributed processing

It is useful to distinguish between several levels of concurrent execution. Multiprogramming allows multiple processes to reside in memory and take turns using the CPU of a single processor; they never truly run in parallel, but rather in an interleaved manner.

Multiprocessing (or multithreading ) uses two or more processors or cores in the same machine to execute one or more processes, achieving true parallelism. Each core can handle a different thread simultaneously, which multiplies performance in many workloads (see the key news from Intel and AMD about CPUs).

Distributed processing takes this a step further: one or more processes run on different computers connected by a network . This introduces additional problems (latency, partial failures, network partitioning), but the conceptual model remains concurrent: many ongoing activities that must be coordinated.

Concurrent programming models: threads, coroutines, event loops, and actors

Concurrency is not always programmed the same way. Several concurrent programming models exist , each offering different abstractions and balancing ease of use, performance, and parallelism in different ways. Among the most common are system threads, coroutines, event loops, and the actor model.

Coroutines

A coroutine is a form of cooperative concurrency within a single operating system thread . Unlike traditional threads, where the OS scheduler decides when to interrupt and resume, coroutines explicitly relinquish control when the scheduler indicates (for example, when waiting for I/O or when a part of the task is completed).

This model is called cooperative because each coroutine must be designed to "behave well": it must relinquish execution at appropriate points to avoid blocking the others. In return, a coroutine's context is much lighter than that of a thread: many coroutines can live within a single system thread, sharing the stack or using very small stacks.

By default, coroutines model concurrency without parallelism : they are all interleaved within a single thread. However, many modern runtimes allow combining them with multiple threads to achieve true parallelism if needed. Concepts such as promises/futures or async/await are often naturally integrated with this model.

Event loop

Another highly influential pattern is the event loop . In this approach, there is generally a single thread that maintains a central loop where events (network data arrivals, timers, user inputs, etc.) are handled and pre-registered callbacks are executed to react to each type of event.

It's called a deferred blocking model because, instead of blocking while waiting for I/O, operations register a handler that will be executed when the event occurs at the end of a loop cycle. This allows multiple concurrent activities to be described on a single thread, avoiding the complexity of inter-thread synchronization and providing very efficient CPU usage.

By default, this model provides concurrency but not parallelism (everything happens on the same thread), although many platforms allow scaling to multiple threads or processes under the hood. A key aspect is that local memory and variable scope behave very similarly to a sequential program, reducing the risk of race conditions.

In the design of event-loop-based APIs, closures or callbacks are frequently used to capture the necessary environment. This can lead to the well-known "pyramid of doom" problem, where nested callbacks make the code difficult to read. Alternatives such as promises/futures or the async/await keywords greatly improve usability.

Actor model

In the actor model, the basic unit of execution is the actor , an entity that encapsulates its own state and behavior and communicates exclusively by sending messages. Each actor has a message queue and processes these messages sequentially, one by one, thus avoiding directly sharing mutable memory.

Actors are, conceptually, extremely lightweight processes managed by a runtime or virtual machine, not directly by the operating system. This allows the creation of thousands or even millions of actors within a single real process, each with its own message box, without the cost of an OS thread per actor.

Execution is typically preemptive: the virtual machine decides when to interrupt one actor to allow another, based on global criteria. If the environment has multiple cores, the runtime can distribute actors among threads or cores relatively transparently, achieving effective parallelism.

An important consequence of this model is that actors do not share mutable memory ; all communication occurs by passing messages (often immutable). This promotes functional purity: an actor only "sees" its own state and the data it receives, making it easier to reason about behavior and exploit safe parallelism. Furthermore, an optimistic approach to error handling is common : if an actor fails due to a transient condition, it restarts without bringing down the entire system.

Which parts of a program can run concurrently

Not all code can be run in parallel or concurrently. There are fragments where the order of execution is critical (for example, initializing a variable before using it), and others where the order is irrelevant (two independent calculations on different data).

To decide whether two blocks of instructions can be executed concurrently, the Bernstein conditions are used . For each instruction set S_{_k} we define:

• L(Sk): set of variables read during the execution of S_k.
• E(Sk): set of variables written (updated) during the execution of S_k.

Two blocks Si _and Sj _can be executed concurrently if three conditions are met: E(Si) ∩ L(Sj) = ∅ , L(Si) ∩ E(Sj) = ∅, and E(Si) ∩ E(Sj) = ∅ . That is, neither block should write to a variable that the other reads or writes. If any of these conditions is violated, executing both blocks simultaneously may change the program's outcome.

Typical problems of concurrent programs

Designing concurrent programs is complicated because the exact order of execution of actions is not fully determined . This non-determinism leads to subtle errors, sometimes difficult to reproduce, that only manifest themselves under certain instruction pairings. Let's look at the most common ones.

Race conditions and critical section

A race condition occurs when the outcome of a computation depends on the relative order in which two or more threads access shared resources, and that order is not under control. If multiple threads modify the same variable without coordination, the final value may be incorrect.

A classic example is the operation x = x + 1 on a shared variable. In a sequential program, this is trivial, but in a multithreaded environment, each increment consists of several instructions: read x, add 1, write the result. If two threads interleave these instructions without mutual exclusion , both can read the same initial value and overwrite each other, losing increments.

In Java, a counter incremented by two threads half a million times each should eventually reach one million, but in the absence of synchronization, different results are observed in each execution. This is because there is no absolute order of operations , but rather a partial order dependent on the scheduler and the specific collation.

To avoid these situations, critical sections are identified —portions of code that access shared resources and should not be executed by more than one thread at a time. Synchronization mechanisms (locks, semaphores, monitors, etc.) ensure that only one thread enters the critical section, just as a traffic light controls access to a narrow intersection.

Violation of mutual exclusion

We speak of a mutual exclusion violation when two or more threads manage to enter the same critical section simultaneously . This breaks the guarantee of exclusive access to the shared resource and leads to undesirable results, such as the previous counter never reaching the expected value.

This type of failure is usually due to incomplete or incorrect synchronization : forgetting to protect access with a lock, using non-atomic instructions, or combining several resources without a clear lock acquisition policy.

Deadlock or interlock

A deadlock occurs when one or more processes are indefinitely waiting for an event that will never happen. Typically, each process in a cycle waits for a resource that the next process has, creating a circular dependency with no way out.

For deadlock to exist in a resource system, four simultaneous conditions are necessary:

• Mutual exclusion: resources are allocated exclusively (not shared).
• Retention and waitingThe processes maintain resources they already possess while requesting additional ones.
• No expropriationResources cannot be forcibly withdrawn from a process; they are only released voluntarily.
• Circular waitThere is a circular chain of processes where each one waits for a resource held by the next one.

Techniques for avoiding or breaking deadlocks are based on negating at least one of these conditions . For example, globally ordering resources to avoid cycles, allowing certain forms of expropriation, or using detection and recovery algorithms. Dijkstra's banker's algorithm is a classic, as is his example of the "dining philosophers."

Indefinite postponement and injustice

Indefinite postponement ( starvation) occurs when a process ready to run or obtain a resource is delayed indefinitely because the scheduling policy never gives it a turn. It is not blocked by a deadlock; there are simply always other processes ahead of it.

This phenomenon is closely linked to unfairness in scheduling algorithms: if the system systematically favors certain processes (by priority, by order of arrival, etc.) without compensating for the accumulated waiting time of the others, it is easy for some to be left "without a candle" forever.

Typical solutions involve implementing aging mechanisms (increasing the priority of those who have been waiting a long time) or treating processes strictly in waiting order. At the design level, this serves as a guideline: when writing concurrent code, one must consider whether the system guarantees that all threads that can progress will eventually do so.

Busy waiting

Another common pitfall is busy waiting : a process enters a loop that constantly checks a condition (for example, the value of a shared variable) and does nothing useful in the meantime. While not functionally incorrect, it wastes CPU and can prevent other processes from receiving enough time.

Ideally, when a process cannot proceed because it is waiting for an event, it should be suspended and the CPU released until the event occurs. This is achieved using blocking primitives that put the thread to sleep (wait, sleep, semaphores, monitor conditions, etc.), instead of letting it run in a loop.

False sharing

In modern architectures with caching, the false sharing problem can arise . This occurs when multiple threads modify different variables located in the same cache line . Although conceptually the data is independent, the cache invalidates and synchronizes the entire line each time one of the threads writes, generating a storm of invalidations that degrades performance.

False sharing does not break the program's correctness, but it causes a significant drop in performance , especially in intensive loops. To mitigate this, techniques such as variable alignment in memory, padding between structure fields, and compiler and runtime options to separate data used by different threads into different cache lines are used.

Safety and life properties in concurrent programs

To determine whether a concurrent program is "well-made," two main sets of properties are typically distinguished: safety and liveness . Both are fundamental to defining correct behavior.

Safety features

Safety properties indicate what the program should never do . They are usually formulated as invariants: conditions that must always remain true throughout execution.

• Mutual exclusionThere should never be more than one process in a critical section at a time.
• Absence of deadlockNo process should be blocked waiting for an event that will never happen.
• Partial CorrectionIf the program terminates, the output obtained must be one of those allowed by the specification.

These properties are usually verified through formal reasoning, systematic testing, or automated verification tools on simplified models of the system.

Properties of life

On the other hand, liveness properties capture what the program should eventually do , that is, that "something good" will eventually happen if there are no external failures.

• JusticeEvery process that is capable of being executed will eventually have its opportunity.
• Reliable communicationEvery message sent ends up being received by the recipient (or at least handled according to the error protocol).
• Completely correctIf the program terminates, it does so with the correct result and does not get stuck in the middle of execution.

Ensuring lifetime properties is often more difficult, as they depend on both the design of the concurrent algorithm and the scheduling policies of the underlying system.

Classic problems of concurrence: readers-writers, producer-consumer, and philosophers

Concurrency theory often relies on canonical problems that serve as a testing ground for synchronization techniques. Several of these have become true classics.

Reader-writer problem

In systems such as databases, shared files, or in-memory structures, it is common to have multiple readers and multiple writers . Readers only read; writers modify. As long as no one is writing, readers can access the data in parallel without issue, but a writer cannot be allowed to modify data while others are reading or writing , to avoid corruption or inconsistent reads.

This problem requires designing algorithms that grant concurrent read access when no writes are in progress, and exclusive access for writers when they need to update. Depending on the solution, readers or writers may be unfairly favored, making it necessary to also consider lifetime properties.

Producer-consumer problem

Another classic scenario involves producers and consumers sharing a buffer (a "store" of items). Producers generate items and deposit them into the buffer; consumers then extract these items for processing. The challenge is to coordinate both sides so that producers don't overflow the buffer and consumers don't try to extract from an empty buffer.

A simple solution is to protect the buffer with mutual exclusion (e.g., binary semaphores) and use counters to block producers when the buffer is full and consumers when it is empty. This approach, while valid, can become cumbersome when there are many actors or the flow becomes complex.

A more structured alternative is to encapsulate the buffer and its synchronization logic in a monitor : an object that offers atomic operations (insert, extract) and internally manages waiting queues, avoiding duplicating synchronization code in producers and consumers.

The Philosophers' Dinner

The philosophers' dinner problem perfectly illustrates deadlocks, race conditions, and starvation. Several philosophers sitting around a table need two forks to eat, but there is only one fork between each pair. If they all try to grab the forks at the same time, it's possible that each one will grab a different fork, and they will all be stuck waiting for the second one: a textbook deadlock.

If the algorithm is not designed well, it can also happen that the same philosopher always goes hungry while others manage to hoard forks repeatedly, which exemplifies injustice and starvation.

The proposed solutions range from a simple round-robin (a system of turns in which they eat one by one), to queues of thinkers where whoever does not get the second fork goes to the end of the queue, to introducing a referee who limits the number of philosophers who can sit at the table simultaneously (for example, a maximum of n−1 if there are n forks), so that there will always be at least one philosopher who can eat and free up resources.

Synchronization and communication methods between processes and threads

To maintain control over access to critical areas and coordinate the execution of concurrent tasks, we have several synchronization and communication primitives . The most commonly used in practice are semaphores, locks, monitors, and explicit message passing.

Traffic lights

A semaphore is a protected integer variable that can only be accessed with two atomic operations: wait/down (decrement and block if the value is zero) and signal/up (increment and, optionally, wake up a blocked process). The operating system guarantees that these operations are indivisible, so they can be used to implement mutual exclusion and other synchronization patterns.

A binary semaphore (only 0 or 1) functions as a lock; a counting semaphore allows a configurable maximum number of threads in a given section. Although very flexible, semaphores are considered low-level abstraction : if misused, they are an endless source of bugs (deadlocks, signal voids, etc.) and force the mixing of synchronization logic with domain logic.

Mutual exclusion through locks

Mutual exclusion is typically implemented using locks (mutual exclusion locks or mutexes). A process that wants to enter a critical section attempts to acquire the lock; if it is free, it enters and marks it as occupied; otherwise, it is blocked until the lock is released.

Conceptually, it's like a lock on the resource : only the person with the key can access the critical section. This pattern is very common in practice, but it requires discipline: the lock must always be released, the same acquisition order must be used when there are multiple resources, and excessively long critical sections that degrade parallelism must be avoided.

Monitors

Monitors take mutual exclusion to a higher level. A monitor is an abstraction that groups shared data and the operations that can be performed on it , ensuring that those operations are executed one by one, never in parallel within the monitor.

In a monitor, when a thread calls one of its methods, it automatically enters a protected region : if another thread is already inside, it must wait its turn. Furthermore, monitors typically offer condition variables (wait/notify, wait/signal) to suspend and wake threads based on certain internal state conditions, simplifying the resolution of problems such as producer-consumer relationships.

Because this technique has a high level of abstraction , it is safer and less prone to errors than the direct use of semaphores or locks scattered throughout the code. Languages ​​such as Java, C#, and various modern runtimes provide native support for monitors and conditions.

Communication through message passing

Another family of techniques dispenses with shared state and is based on exchanging messages between processes or threads. Each message typically has a header (sender and receiver identifiers, message type, size) and a body with the necessary data.

There are several addressing modes: in direct addressing , the sender explicitly indicates who the receiver is, and both know each other. In implicit addressing , the sender specifies the receiver, but the receiver doesn't necessarily know who is sending the message. Finally, in indirect addressing , messages are deposited into mailboxes managed by specific processes or by the operating system; the interested processes read from the mailbox to determine if they can access a particular critical section or process a certain event.

Synchronization can be blocking (the sender waits for the receiver to receive, or the receiver waits for a message) or non-blocking. Choosing which combinations to use is crucial: a poor choice can create new race conditions or deadlocks instead of resolving them.

Process planning and context changes

Process scheduling is the strategy by which the operating system decides which process or thread runs at any given time, sharing the CPU among all those in memory. It relies on managing queues of ready processes and applying scheduling algorithms with objectives such as maximizing CPU utilization and minimizing delays (see the new kernel features ).

There are usually three levels of scheduling:

• Short term: decides which ready process gets the CPU next. Directly controls the immediate allocation of processor time.
• Medium termThe CPU scheduler, also called the CPU scheduler or dispatcher, decides which processes should remain in main memory and which can be swapped to optimize overall performance. dispatch latency It is the time it takes the system to stop one process and start another.
• Long term: regulates which processes are accepted into the system and which are removed from memory, affecting the global multiprogramming level (how many processes are active at the same time).

Classic scheduling algorithms include FCFS (First-Come, First-Served), SJF (Shortest-Job-First), priority scheduling , Round Robin , and multilevel queuing . Each offers different trade-offs in terms of response time, throughput, fairness, and complexity.

To switch between processes, context switches are necessary : ​​the system saves the contents of the outgoing process's registers, stack pointer, and program counter to its PCB, and restores those of the incoming process. This mechanism allows a process to resume exactly where it left off, as if nothing had happened in between.

Concurrent systems verification tools

Since the behavior of a concurrent system can vary greatly depending on the order of events, it is common to use formal models and verification tools to help detect deadlocks, race conditions, and other violations of desired properties before deploying the actual system.

Petri nets

Petri nets are a graphical model for describing concurrent systems. They are represented as a directed, bipartite graph with two types of vertices: places (conditions) and transitions (events). Places contain tokens that indicate which conditions are met in a given state.

A transition is enabled if all its entry locations have at least one token. When a transition is triggered, it consumes one token from each entry location and adds tokens to its exit locations. Starting with an initial marking (token distribution), it is possible to study properties such as whether the system is live (every transition can eventually be triggered) or bounded/safe (tokens do not accumulate infinitely in one location).

CSP (Communicating Sequential Processes)

CSP is a mathematical theory for specifying and reasoning about systems composed of sequential processes that communicate through channels. Its formal semantics allow for the description of interaction patterns, the detection of deadlocks or livelocks , and the verification that a design meets certain safety and liveness properties.

In CSP, processes connect through unidirectional channels and synchronize using a rendezvous mechanism: writing and reading a message on a channel are considered joint actions that occur simultaneously, avoiding certain forms of race conditions associated with traditional shared state.

FDR and JCSP

FDR (Failures-Divergence Refinement) is a CSP-based tool that automatically checks whether a system model meets certain properties, also expressed as CSP processes. The idea is to verify if the actual system is a refinement (not introducing new faults or divergences) of the desired abstract model, detecting, for example, security violations or potential deadlocks.

JCSP , on the other hand, is a Java library that directly implements CSP concepts: processes, channels, synchronization, etc. It simplifies writing concurrent programs in Java without resorting to locks and direct memory sharing, relying on secure and well-defined communication primitives such as channels, barriers, timers, and other high-level components.

Taken together, this entire ecosystem of models, classic problems, synchronization primitives, and verification tools provides a solid framework for designing and analyzing reliable, efficient, and scalable concurrent programs capable of taking advantage of both task interleaving and the parallelism available in modern hardware.