Complete tutorial of the journalctl command in Linux

If you administer GNU/Linux systems daily, sooner or later you'll end up struggling with the system logs, services, and kernelPreviously, you had to jump between files in /var/logBut with the arrival of systemd — manage services with systemd — the game has changed: now almost everything happens through the newspaper or journaland the key to controlling it is the command journalctl.

In this guide you will see, in great detail and with practical examples, how to use journalctl to query, filter, track in real time and clean logsThis includes how to limit the disk space they occupy and how to take advantage of their advanced filters (by service, PID, user, priority, time, kernel, etc.). The idea is that you end up with a complete overview, without having to search for pieces of information in a thousand different places.

What is journalctl and what role does it play in systemd?

journalctl is the line utility of commands which allows you to view the systemd logIt is part of the ecosystem of systemdwhich is the init system and service manager used today by distributions such as Ubuntu, Debian, Fedora, CentOS/RHEL 7+, Arch Linux and many others.

The component responsible for collecting the events is the daemon systemd-journald, which collects messages from kernel, initrd, services, systemd units, user processes, and classic syslog messagesAll of that is stored in a binary format (not plain text), which facilitates very fast searches, advanced filtering and multiple output formats.

Unlike the traditional approach of having several scattered log files, the journal acts as centralized system registration pointThis allows, for example, viewing Nginx and PHP-FPM messages on a single timeline, or combining kernel messages with those from a specific service to troubleshoot a problem. hardware that affects an application.

Another advantage of the binary format is that The same data can be entered in different formats Depending on your needs: classic syslog style, JSON for use with external tools, export format for backups, or formats with more precise timestamps. There's no need to convert files because the log internally stores events with structured metadata.

The systemd journal can coexist with a classic syslog server (rsyslog, syslog-ng, etc.) or even replace it. You can continue sending logs to a central server via syslog and, at the same time, use journalctl for local debugging with all the power of filters and metadata that systemd offers.

Preliminary checks: time zone and system time

Since the journal allows timestamps to be displayed in local time or UTCIt's a good idea to have your time zone correctly configured so you don't get confused when correlating events. This is done using the tool... timedatectlwhich is also part of systemd.

To list all available time zones on your system, you can run timedatectl list-timezonesand thus locate the one that corresponds to your location. Once located, you define it with the command sudo timedatectl set-timezone ZONA/HORARIA, for example Europe/Madrid for the Spanish capital.

If you want to check that everything is working correctly, simply run timedatectl statuswhere you will see data such as Local time, Universal Time (UTC), active time zone, and NTP synchronization status. From there, journalctl It will display dates in your local time by default, making it much easier to read.

First contact: view all journal entries

The most basic way to use it is to launch journalctl without argumentsBy doing so, the command will display all available entries in the journal, from oldest to newest, using a paginator (typically less) so that you can move around the exit.

In that list you will see messages from process of Bootkernel, services, application errors, user sessions and so on. If the system has been running for weeks or months, it's perfectly normal to find tens or hundreds of thousands of linesSo the first thing you'll understand is the need to filter.

The default format is very similar to the classic one syslog: date, hostname, process, PID, and message. The important difference is that, although you see it in text, internally Everything is structured with fields and metadata which you can access later with different filters.

If at any point you want to see dates in UTC instead of local time, you can add the parameter to the command. --utc, and all timestamps will be adjusted accordingly without affecting the original content.

Filter by start times and time windows

One of the most useful things about journalctl It allows you to limit the records by boot sessions or by specific time rangesThis is especially useful on servers with high uptime or when you want to review exactly what happened during a specific reboot.

Show only the current startup

To focus on what has happened since the last system reboot, you can use the indicator -bThis parameter indicates to journalctl that shows only the entries collected from the current boot:

Command: journalctl -b

When you don't filter by startup and several restarts appear in the log, the journal inserts lines like -- Reboot -- to mark the separations between sessions. They are very useful for orientation, but in day-to-day practice it is usual to limit yourself to the startup in progress to debug current problems.

View previous startups and enable persistent logging

If you want to investigate problems that occurred in previous bootsYou will need the system to have journal persistence enabled. In some distributions it is enabled by default; in others, you have to create the directory /var/log/journal or tap the settings.

The simplest way to enable storage persistent is creating said directory with sudo mkdir -p /var/log/journal or edit /etc/systemd/journald.conf and establish in the section the option Storage=persistentFrom that point on, the records will be retained between reboots.

To list all the starts that the journal has a record of, you can use journalctl --list-bootsYou will see a table with a relative index (0 for the current one, -1 for the previous one, etc.), the unique boot ID, and the date range for each session.

If you want to access the previous boot, you can use that relative index, for example journalctl -b -1or directly pass the full boot ID, which is useful when you want to share an exact reference in documentation or tickets: journalctl -b ID_DE_BOOT.

Define custom time windows

Besides for outbursts, journalctl allows filtering by arbitrary time rangesThis is essential for servers that are frequently restarted. The following options are used for this purpose: --since y --until, indicating a start time and, optionally, an end time.

The standard date and time format is YYYY-MM-DD HH:MM:SSHowever, it is quite flexible: if you omit the date, the current day is assumed; if you omit the time, the current time is used. midnight (00:00:00)You can also leave out the seconds.

Typical examples would be journalctl --since "2015-01-10 17:15:00" to show everything that has happened since that date and time, or journalctl --since "2015-01-10" --until "2015-01-11 03:00" to precisely define a specific interval that may coincide with an incident that has been reported to you.

The journal also understands related keywords , the yesterday, today, tomorrow o nowFurthermore, it accepts expressions like "1 hour ago"which allows you to query things like: journalctl --since 09:00 --until "1 hour ago" to review a dynamic time window without having to calculate exact dates.

Filter by service, processes, users, and other fields

Once you've mastered time filters, the next step is focus the logs on specific services or components. this is where journalctl It becomes especially powerful compared to reviewing text files directly.

Filter by systemd unit

You'll probably use the unit filter the most, using the parameter -uThis allows you to see only events related to a specific serviceFor example, to see what's happening with Nginx you can run:

Example: journalctl -u nginx.service

It is common to combine this with time filters, for example journalctl -u nginx.service --since today to see only what happened today. This way you avoid noise from previous days and focus on the service's recent behavior.

One of the great advantages of journaling is that you can interleave records from multiple units in a single timeline. If you have Nginx communicating with PHP-FPM, you can do something like journalctl -u nginx.service -u php-fpm.service --since today and you will have the logs of both services ordered chronologically, very useful for detecting communication problems between them.

Filter by PID, UID, and GID

In services that generate many child processes, or when you want to investigate the behavior of a specific process, it is very practical to filter by Process ID (PID)This is done using the field _PIDwhich the journal automatically adds as metadata:

Filter: journalctl _PID=8088

Similarly, if you want to see all entries generated by a specific user or groupYou can use the fields _UID y _GIDFor example, if your web server runs as www-dataFirst, you obtain its UID with id -u www-data and then you filter with something like journalctl _UID=33 --since today.

The underscore prefix indicates that they are fields added by journald using system informationnot simply data that the process has written. This makes it possible to perform very precise searches without requiring the application to "behave" by formatting its messages.

If you have any questions about which fields exist, you can consult the manual page. systemd.journal-fieldsAnd if you want to see what distinct values ​​a field has in your journal, you can use journalctl -F NOMBRE_CAMPO, for example journalctl -F _GID to list all the GIDs that appear in the logs.

Filter by executable path

Another interesting option is to filter by path of an executableIf you move on to journalctl A path as an argument will show you all entries related to that binary, for example:

By route: journalctl /usr/bin/bash

It's usually preferable to work with systemd units when they exist, because that way you also get child process events and additional metadata, but for programs that do not have their own unit This method can be very useful.

View only kernel messages

The messages we traditionally obtained with dmesg They are also stored in the journal. To filter them without further explanation, you can use the options. -k o --dmesg:

Kernel: journalctl -k

By default, kernel messages are displayed from current bootBut you can use the same boot selection options as before, for example journalctl -k -b -5 to view kernel messages from five boot times ago.

Filter by priority or severity level

When the log is full of information, often you're only interested in seeing errors, alerts, or critical failures. journalctl allows filtering by priority using the option -p, accepting both symbolic names and numerical values.

The levels implemented by the journal follow the standard syslog scale, from most to least important: 0: emergency, 1: alert, 2: critical, 3: error, 4: warning, 5: notice, 6: info, 7: debugIf you run something like journalctl -p err -b, you'll see all error-level or higher events of the current start.

You can use either the number or the name, for example journalctl -p 4 o journalctl -p warningAnd messages with that level and all those of higher priority will always be displayed, saving you from having to combine several filters.

Adjust the way records are displayed

In addition to filtering which entries you want to see, journalctl allows you to modify the format and presentation of the output to adapt it to what you need at any given time, whether it's interactive reading or automatic processing by scripts.

Shorten or show the full entry

By default, the paginator displays the entire log lines, even if they overflow to the right; you can move the cursor horizontally to see the whole content. If you prefer that The lines are cut off and appear with ellipses at the endYou can use the option --no-full.

At the other extreme, if you want to journalctl Show all data without hiding non-printable characters, you can add the parameter -aThis is useful when working with escapes or messages with unusual formats.

Bypass the pager and dump to standard output

When the intention is channel the logs to other tools (grep, awk, jq…) or redirect them to a file; the default pager is annoying. To disable it, simply use the option --no-pagerand everything will be printed directly through standard output.

Thanks to this, you can easily chain commands like journalctl -u nginx --since today --no-pager | grep 500 or dump a certain part of the journal to a file with journalctl > mensajes.log to analyze it calmly later.

Choose the output format: syslog, JSON, export, verbose…

The option -o allows to select among several output formats depending on what you're going to do with that information. Some of the most useful formats are:

• shorts: default format, classic syslog style.
• short-iso: same as the previous one but with readable ISO 8601 timestamps.
• short-monotonic: adds the monotonic timestamp, useful for viewing relative times from the start.
• short-precise: same as the standard but with microsecond precision.
• cat: displays only the message field, without metadata.
• json: one JSON input per line, great for consuming with jq or other tools.
• json-pretty: JSON formatted with indents for human reading.
• json-sse: intended for server-sent events (Server-Sent Events).
• export: suitable binary format for export or back up the journal.
• verbose: sample all fields and metadata associated with each event, including inmates.

For example, to extract the Nginx logs from the current boot in JSON format ready for parsing, you could use something like journalctl -b -u nginx -o jsonOr if you want to give them a legible preview before passing them to another tool, journalctl -b -u nginx -o json-pretty.

Follow the logs in real time and see the latest entries

In daily administrative work, it is very common to want monitor what is happening at that very momentFor example, while restarting a service, reproducing an error, or launching a deployment. journalctl It includes as standard features similar to tail.

Show only the most recent entries

The option -n works exactly the same as tail -n: displays the last N entries in the log. If you don't specify a number, the following are displayed by default: Last 10 lines:

Latest: journalctl -n

If you want to change that amount, simply add the number below, for example journalctl -n 20 To view the 20 most recent entries. Combined with other filters (by service, priority, etc.) it's perfect for getting a quick overview of the status of something specific.

Follow the journal live in tails-f style

To monitor the logs in real time, as they are being writtenYou can use the option -f, Abbreviation of followexactly the same as in tail -f:

Monitor: journalctl -f

The interesting thing is that you can combine it with filters by service, priority, or time. For example, you can only track errors in a particular service with a command of the type journalctl -u apache2 -p err -fAnd you'll see new entries appear as they are generated.

Disk space management: journal size and deletion of old logs

Journaling is very convenient, but if you let it grow unchecked it can easily start to... occupy a significant amount of spaceespecially on servers with high traffic. Fortunately, journalctl y systemd-journald They offer several ways to control their size.

Check how much space the logs occupy

To see at a glance how much disk space the journal files are currently using, you can run:

Disc report: journalctl --disk-usage

The exit will indicate something like this «Journals take up 8.0M on disk», clearly showing you the current consumption. This figure takes into account both storage persistent in /var/log/journal like the volatile in /run when there is no persistence.

Delete old records by size or by time

If you decide that the journal is taking up too much space, you can reduce it in two main ways using vacuum parameters (available from systemd 218 onwards): by total size or by age.

With option --vacuum-size you establish a maximum size limit for the journal file set. journald It will delete older entries until the total reaches the limit you specify, for example:

Limit size: sudo journalctl --vacuum-size=1G

The other way is to use --vacuum-timewhich deletes all entries prior to a certain time threshold. For example, if you only want to keep the logs from the last yearYou could run:

Limit age: sudo journalctl --vacuum-time=1years

Both methods can be combined in the configuration of journald so that the system self-manage preventively without you having to run when the disk fills up.

Configure permanent limits in journald.conf

To fine-tune how the journal grows, you need to edit the file /etc/systemd/journald.confThere you will find several key options that affect both persistent and volatile storage in /run:

• SystemMaxUse=: limits the maximum disk space that the journal can use in persistent storage.
• SystemKeepFree=: indicates how much space the journal should leave free for other uses when writing to persistent storage.
• SystemMaxFileSize=: defines the maximum size of each individual file in the persistent journal before rotation.
• RuntimeMaxUse=: analogous to SystemMaxUse but for volatile storage in /run.
• RuntimeKeepFree=: equivalent to SystemKeepFree but for the volatile area.
• RuntimeMaxFileSize=: maximum size of individual journal files in volatile storage before rotation.

Values ​​accept suffixes such as K, M, G or T (kilobytes, megabytes, gigabytes, terabytes), so you can put things like SystemMaxUse=500M No problem. After changing the settings, remember restart service with something like sudo systemctl restart systemd-journald for the changes to take effect.

Keep in mind that SystemMaxFileSize y RuntimeMaxFileSize These are indicative values: the system will try to adjust the file sizes to these limits, but they will not always match exactly, so when interpreting the number of files after a vacuum, don't be surprised if some differ slightly from the expected number.

Other useful uses and best practices with journalctl

In addition to all the above, journalctl offers some extra options that are worth having on hand for day-to-day system administration, both at the level of a normal user and root or privileged groups (wheel, adm, systemd-journal).

For example, it is possible filter the journal by specific user using _UIDAs we have seen, combining time filters with service and priority, or even using the option --grep built-in for searching text strings without having to resort to a grep external.

You can also redirect journal output directly to a file with something as simple as journalctl > mensajes.logThis is useful for sharing a log extract with a colleague or attaching it to a report without granting access to the server.

For those who manage multi-user systems, it is useful to remember that By default, each user can see their own journals.However, global access to system logs and other users' logs is restricted to administrators, precisely to prevent leaks of sensitive information.

Finally, it's good to keep in mind that systemd y journalctl They are highly integrated with other ecosystem utilities, such as systemctl to list and manage services o timedatectl to adjust the time. Taking advantage of this integration makes your life much simpler than mixing old and new systems.

With all this arsenal of options, filters, and settings, journalctl becomes a central tool for monitoring, debugging, and maintaining the health of any systemd-based Linux systemallowing you to go from chasing scattered logs to having a structured and manageable view of everything that happens on the machine.