Understanding Pipes in Unix and Linux – Learn Operating Systems

One of the most elegant features of Unix and Unix-like operating systems is the pipe. Pipes allow processes to communicate with one another by passing data directly between programs. This simple concept has been a cornerstone of Unix design since the early 1970s and remains widely used in Linux, BSD, and macOS today.

What Is a Pipe?

A pipe is a mechanism that allows the output of one process to become the input of another process.

Think of a pipe as a digital pipeline. One program writes data into one end, and another program reads that data from the other end.

Instead of storing temporary data in a file, processes can exchange information directly through the pipe.

The Unix Philosophy

Pipes are closely tied to the Unix philosophy:

Write programs that do one thing well and work together.

Rather than creating one large program that performs many tasks, Unix encourages developers to build small tools that can be combined.

Pipes make this possible.

For example, one program can generate data, another can sort it, and a third can filter it, all connected through pipes.

Pipes on the Command Line

The pipe symbol is:

A simple example is:

ls | sort

Here’s what happens:

ls generates a list of files.
The operating system creates a pipe.
The output of ls is sent into the pipe.
sort reads the data from the pipe.
sort displays the sorted results.

The two programs run as separate processes but communicate through the pipe.

Another Example

Suppose you want to count the number of files in a directory:

ls | wc -l

In this case:

ls produces a list of files.
The pipe transfers the list.
wc -l counts the lines received.

The result is the number of files displayed.

How Pipes Work Internally

When a process creates a pipe, the operating system allocates a temporary buffer in memory.

The pipe has two ends:

Write End

The sending process writes data into the pipe.

Read End

The receiving process reads data from the pipe.

The operating system manages the buffer and ensures that data flows correctly between the two processes.

A simplified view looks like this:

Process A ---> [ Pipe Buffer ] ---> Process B

The processes do not need to know how the data is transferred. The operating system handles the details.

Creating Pipes in C

Unix-like operating systems provide the pipe() system call.

A simplified example looks like:

int fd[2];
pipe(fd);

After the call:

fd[0] is the read end.
fd[1] is the write end.

One process writes to fd[1], while another reads from fd[0].

This mechanism forms the basis of many forms of interprocess communication.

Parent and Child Processes

Pipes are commonly used between parent and child processes.

A typical sequence is:

Create a pipe.
Create a child process using fork().
Parent writes data to the pipe.
Child reads data from the pipe.

For example:

Parent Process
      |
      | writes
      V
    Pipe
      |
      | reads
      V
 Child Process

This allows related processes to exchange information efficiently.

Blocking Behavior

Pipes have limited buffer space.

If the buffer becomes full:

The writing process may pause until space becomes available.

If the buffer is empty:

The reading process may pause until new data arrives.

This behavior allows the operating system to synchronize communication automatically.

Anonymous Pipes

The pipes commonly used with shell commands are known as anonymous pipes.

Characteristics include:

Temporary
Exist only while processes are running
Usually used between related processes

Examples include:

ls | grep txt

and

ps aux | grep apache

Named Pipes (FIFOs)

Unix systems also support named pipes, sometimes called FIFOs (First In, First Out).

Unlike anonymous pipes, named pipes appear in the file system.

They can be created using:

mkfifo mypipe

Once created, unrelated processes can communicate through the named pipe.

Named pipes are useful when processes need a permanent communication channel.

Advantages of Pipes

Pipes provide several benefits:

Fast communication between processes
Reduced need for temporary files
Simple design
Efficient use of memory
Support for modular software development

These advantages helped make Unix systems powerful and flexible.

Pipes in Modern Operating Systems

Pipes remain an important part of:

Linux
FreeBSD
OpenBSD
NetBSD
macOS
Other Unix-like systems

Although modern systems also support sockets, message queues, and shared memory, pipes remain one of the simplest and most widely used forms of interprocess communication.

Conclusion

Pipes are one of the defining features of Unix and Unix-like operating systems. They allow separate processes to communicate by passing data through a memory buffer managed by the operating system. Whether used on the command line or within system software, pipes embody the Unix philosophy of combining small programs to accomplish complex tasks.

Understanding pipes provides valuable insight into how operating systems enable processes to work together and forms an important foundation for studying process management, interprocess communication, and operating system design.