1
0
mirror of https://github.com/systemd/systemd synced 2025-10-03 18:54:45 +02:00

Compare commits

..

7 Commits

Author SHA1 Message Date
Luca Boccassi
827ef318ec test: send trailing newlines in notify messages in TEST-50-DISSECT
It seems the failing test in https://github.com/systemd/systemd/issues/37626
is due to MONOTONIC_USEC= being somehow lost. Add a trailing newline when
sending messages with socat, hopefully ensuring it is delivered and read.
2025-07-31 05:43:56 +09:00
Yu Watanabe
700c60dff1
Double journal field hash table size (#38416) 2025-07-31 05:43:21 +09:00
ssoss
1e55da38aa network: add new DHCPv6 message types and options
Gives enum definitions for the DHCPv6 options from RFC9527 and RFC 9686
and the new message types defined in RFC 9686.
2025-07-31 05:41:53 +09:00
Yu Watanabe
0d229ef102 resolve: bump *_SEARCH_DOMAIN_MAX to 1024
As prompted by #38393, search domains may be large when a complicated
network setting is used, especially when VPN is used. Let's bump the
limit to 1024.

Note, this does not bump the maximum number of DNS servers, as setting
thousands of DNS servers is spurious and mostly meaningless. Let's keep
the maximum for a while until someone requests to also bump them.

Continuation of b950ee06e647d753db2047c1bb5345c94391be3e.

Closes #38393.
2025-07-30 18:57:09 +01:00
Zbigniew Jędrzejewski-Szmek
e8962d77ac journal: treble field hash table size
As discussed in https://github.com/systemd/systemd/issues/38399, "ordinary"
systems can have the field table with a large number of values, causing journal
rotation to occur early. For example, audit generates a log of fields:
$ journalctl --fields | rg -c '^_?AUDIT'
114

It seems that the "structured log" capabilities of the journal are being use
more than in the past. Looking at some journal files on my system, it seems
the field hash table field is quite high in many cases:
$ build/test-journal-dump /var/log/journal/*/* | rg 'table fill'
Data hash table fill: 15.1%
Field hash table fill: 69.1%
Data hash table fill: 4.9%
Field hash table fill: 32.4%
Data hash table fill: 10.2%
Field hash table fill: 34.2%
Data hash table fill: 9.9%
Field hash table fill: 37.2%
Data hash table fill: 26.8%
Field hash table fill: 21.9%
Data hash table fill: 35.6%
Field hash table fill: 22.8%
Data hash table fill: 25.5%
Field hash table fill: 54.1%
Data hash table fill: 3.4%
Field hash table fill: 43.8%
Data hash table fill: 75.0%
Field hash table fill: 70.3%
Data hash table fill: 75.0%
Field hash table fill: 63.1%
Data hash table fill: 75.0%
Field hash table fill: 74.2%
Data hash table fill: 35.6%
Field hash table fill: 43.2%
Data hash table fill: 35.5%
Field hash table fill: 75.4%
Data hash table fill: 75.0%
Field hash table fill: 59.8%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 16.9%
Field hash table fill: 76.3%
Data hash table fill: 18.1%
Field hash table fill: 76.9%
Data hash table fill: 75.0%
Field hash table fill: 42.0%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 32.1%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.5%
Data hash table fill: 9.6%
Field hash table fill: 53.8%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 35.6%
Field hash table fill: 75.1%
Data hash table fill: 33.6%
Field hash table fill: 50.2%
Data hash table fill: 75.0%
Field hash table fill: 26.7%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 31.8%
Data hash table fill: 75.0%
Field hash table fill: 18.9%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 20.1%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 30.9%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 25.2%
Data hash table fill: 75.0%
Field hash table fill: 39.3%
Data hash table fill: 50.2%
Field hash table fill: 75.1%
Data hash table fill: 75.0%
Field hash table fill: 61.9%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 75.0%
Field hash table fill: 58.6%
Data hash table fill: 48.9%
Field hash table fill: 79.6%
Data hash table fill: 75.0%
Field hash table fill: 71.5%
Data hash table fill: 75.0%
Field hash table fill: 60.1%
Data hash table fill: 31.4%
Field hash table fill: 75.7%
Data hash table fill: 27.0%
Field hash table fill: 69.4%
Data hash table fill: 28.9%
Field hash table fill: 76.6%
Data hash table fill: 60.2%
Field hash table fill: 79.9%
Data hash table fill: 8.8%
Field hash table fill: 78.7%
Data hash table fill: 5.8%
Field hash table fill: 61.3%
Data hash table fill: 75.0%
Field hash table fill: 64.0%
Data hash table fill: 61.4%
Field hash table fill: 63.4%
Data hash table fill: 29.7%
Field hash table fill: 61.9%
Data hash table fill: 18.9%
Field hash table fill: 30.9%
Data hash table fill: 1.4%
Field hash table fill: 22.2%
Data hash table fill: 0.4%
Field hash table fill: 13.5%
Data hash table fill: 2.6%
Field hash table fill: 37.5%
Data hash table fill: 1.3%
Field hash table fill: 23.4%
Data hash table fill: 0.6%
Field hash table fill: 15.3%
Data hash table fill: 18.7%
Field hash table fill: 33.9%
Data hash table fill: 7.4%
Field hash table fill: 37.5%
Data hash table fill: 20.2%
Field hash table fill: 44.1%
Data hash table fill: 1.3%
Field hash table fill: 33.0%
Data hash table fill: 75.0%
Field hash table fill: 19.2%
Data hash table fill: 42.2%
Field hash table fill: 23.4%
Data hash table fill: 1.6%
Field hash table fill: 87.1%
Data hash table fill: 0.1%
Field hash table fill: 98.8%
Data hash table fill: 0.2%
Field hash table fill: 128.8%
Data hash table fill: 15.4%
Field hash table fill: 31.2%
Data hash table fill: 7.4%
Field hash table fill: 22.5%
Data hash table fill: 10.5%
Field hash table fill: 38.7%
Data hash table fill: 2.8%
Field hash table fill: 18.0%
Data hash table fill: 1.5%
Field hash table fill: 15.9%
Data hash table fill: 0.0%
Field hash table fill: 7.5%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.2%
Field hash table fill: 15.6%
Data hash table fill: 0.1%
Field hash table fill: 11.7%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.0%
Field hash table fill: 6.6%
Data hash table fill: 1.4%
Field hash table fill: 18.0%
Data hash table fill: 0.7%
Field hash table fill: 16.8%
Data hash table fill: 1.1%
Field hash table fill: 18.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.1%
Field hash table fill: 10.8%
Data hash table fill: 0.4%
Field hash table fill: 11.1%

Since filling of the field hash table to 75% normally causes file rotation,
let's double the default to make rotation happen less often.
We'll use 11kB more for the hash table, which should be fine, considering
that journal files are usually at least 8 MB.

Closes https://github.com/systemd/systemd/issues/38399.
2025-07-30 15:57:19 +02:00
Zbigniew Jędrzejewski-Szmek
fbbcd0edef journal: store counts, not byte sizes, in table size constants
It's easier to think about the size in "objects", not bytes. Let's convert to
bytes at the last moment.

Also drop some of the pointless size suffixes. In general, it's the size of the
variable that matters, not the constant that is written to it.

No functional change.
2025-07-30 15:51:00 +02:00
Zbigniew Jędrzejewski-Szmek
58ecb72bcf test-journal-dump: dump the headers of journal files
We have journal_file_print_header(), but it's not exposed anywhere in
a way that it is easy to call.
2025-07-30 15:51:00 +02:00
10 changed files with 88 additions and 29 deletions

View File

@ -210,6 +210,10 @@ bool dhcp6_option_can_request(uint16_t option) {
case SD_DHCP6_OPTION_V6_DOTS_ADDRESS:
case SD_DHCP6_OPTION_IPV6_ADDRESS_ANDSF:
case SD_DHCP6_OPTION_V6_DNR:
case SD_DHCP6_OPTION_REGISTERED_DOMAIN:
case SD_DHCP6_OPTION_FORWARD_DIST_MANAGER:
case SD_DHCP6_OPTION_REVERSE_DIST_MANAGER:
case SD_DHCP6_OPTION_ADDR_REG_ENABLE:
return true;
default:
return false;

View File

@ -98,6 +98,8 @@ typedef enum DHCP6MessageType {
DHCP6_MESSAGE_DISCONNECT = 33, /* RFC 8156 */
DHCP6_MESSAGE_STATE = 34, /* RFC 8156 */
DHCP6_MESSAGE_CONTACT = 35, /* RFC 8156 */
DHCP6_MESSAGE_ADDR_REG_INFORM = 36, /* RFC 9686 */
DHCP6_MESSAGE_ADDR_REG_REPLY = 37, /* RFC 9686 */
_DHCP6_MESSAGE_TYPE_MAX,
_DHCP6_MESSAGE_TYPE_INVALID = -EINVAL,
} DHCP6MessageType;

View File

@ -275,6 +275,10 @@ libsystemd_tests += [
'sources' : files('sd-journal/test-journal-append.c'),
'type' : 'manual',
},
{
'sources' : files('sd-journal/test-journal-dump.c'),
'type' : 'manual',
},
{
'sources' : files('sd-journal/test-journal-verify.c'),
'timeout' : 90,

View File

@ -46,11 +46,11 @@
#include "user-util.h"
#include "xattr-util.h"
#define DEFAULT_DATA_HASH_TABLE_SIZE (2047ULL*sizeof(HashItem))
#define DEFAULT_FIELD_HASH_TABLE_SIZE (333ULL*sizeof(HashItem))
#define DEFAULT_DATA_HASH_TABLE_SIZE 2047U
#define DEFAULT_FIELD_HASH_TABLE_SIZE 1023U
#define DEFAULT_COMPRESS_THRESHOLD (512ULL)
#define MIN_COMPRESS_THRESHOLD (8ULL)
#define DEFAULT_COMPRESS_THRESHOLD 512U
#define MIN_COMPRESS_THRESHOLD 8U
/* This is the minimum journal file size */
#define JOURNAL_FILE_SIZE_MIN (512 * U64_KB) /* 512 KiB */
@ -1286,15 +1286,14 @@ static int journal_file_setup_data_hash_table(JournalFile *f) {
beyond 75% fill level. Calculate the hash table size for
the maximum file size based on these metrics. */
s = (f->metrics.max_size * 4 / 768 / 3) * sizeof(HashItem);
if (s < DEFAULT_DATA_HASH_TABLE_SIZE)
s = DEFAULT_DATA_HASH_TABLE_SIZE;
s = MAX(f->metrics.max_size * 4 / 768 / 3,
DEFAULT_DATA_HASH_TABLE_SIZE);
log_debug("Reserving %"PRIu64" entries in data hash table.", s / sizeof(HashItem));
log_debug("Reserving %"PRIu64" entries in data hash table.", s);
r = journal_file_append_object(f,
OBJECT_DATA_HASH_TABLE,
offsetof(Object, hash_table.items) + s,
offsetof(Object, hash_table.items) + s * sizeof(HashItem),
&o, &p);
if (r < 0)
return r;
@ -1302,7 +1301,7 @@ static int journal_file_setup_data_hash_table(JournalFile *f) {
memzero(o->hash_table.items, s);
f->header->data_hash_table_offset = htole64(p + offsetof(Object, hash_table.items));
f->header->data_hash_table_size = htole64(s);
f->header->data_hash_table_size = htole64(s * sizeof(HashItem));
return 0;
}
@ -1319,19 +1318,19 @@ static int journal_file_setup_field_hash_table(JournalFile *f) {
* number should grow very slowly only */
s = DEFAULT_FIELD_HASH_TABLE_SIZE;
log_debug("Reserving %"PRIu64" entries in field hash table.", s / sizeof(HashItem));
log_debug("Reserving %"PRIu64" entries in field hash table.", s);
r = journal_file_append_object(f,
OBJECT_FIELD_HASH_TABLE,
offsetof(Object, hash_table.items) + s,
offsetof(Object, hash_table.items) + s * sizeof(HashItem),
&o, &p);
if (r < 0)
return r;
memzero(o->hash_table.items, s);
memzero(o->hash_table.items, s * sizeof(HashItem));
f->header->field_hash_table_offset = htole64(p + offsetof(Object, hash_table.items));
f->header->field_hash_table_size = htole64(s);
f->header->field_hash_table_size = htole64(s * sizeof(HashItem));
return 0;
}

View File

@ -0,0 +1,46 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#include "errno-util.h"
#include "journal-file.h"
#include "log.h"
#include "main-func.h"
#include "pager.h"
#include "strv.h"
static int run(int argc, char *argv[]) {
int r = 0;
unsigned n = 0;
_cleanup_(mmap_cache_unrefp) MMapCache *m = mmap_cache_new();
assert_se(m);
pager_open(/* flags= */ 0);
STRV_FOREACH(s, strv_skip(argv, 1)) {
JournalFile *f = NULL;
int k = journal_file_open(
/* fd= */ -EBADF,
*s,
O_RDONLY,
/* file_flags= */ 0,
0666,
/* compress_threshold_bytes= */ UINT64_MAX,
/* metrics= */ NULL,
m,
/* template= */ NULL,
&f);
if (k < 0)
RET_GATHER(r, log_error_errno(k, "Failed to open %s, continuing: %m", *s));
if (n++ > 0)
puts("");
journal_file_print_header(f);
journal_file_close(f);
}
return r;
}
DEFINE_MAIN_FUNCTION(run);

View File

@ -4,7 +4,7 @@
#include "list.h"
#include "resolved-forward.h"
#define DELEGATE_SEARCH_DOMAINS_MAX 256
#define DELEGATE_SEARCH_DOMAINS_MAX 1024
#define DELEGATE_DNS_SERVERS_MAX 256
/* A DnsDelegate object is used to manage additional, explicitly configured unicast DNS lookup scopes,

View File

@ -9,7 +9,7 @@
#include "resolve-util.h"
#include "resolved-forward.h"
#define LINK_SEARCH_DOMAINS_MAX 256
#define LINK_SEARCH_DOMAINS_MAX 1024
#define LINK_DNS_SERVERS_MAX 256
typedef struct LinkAddress {

View File

@ -15,7 +15,7 @@
#include "resolved-etc-hosts.h"
#include "resolved-forward.h"
#define MANAGER_SEARCH_DOMAINS_MAX 256
#define MANAGER_SEARCH_DOMAINS_MAX 1024
#define MANAGER_DNS_SERVERS_MAX 256
typedef struct Manager {

View File

@ -165,9 +165,13 @@ enum {
SD_DHCP6_OPTION_SLAP_QUAD = 140, /* RFC 8948 */
SD_DHCP6_OPTION_V6_DOTS_RI = 141, /* RFC 8973 */
SD_DHCP6_OPTION_V6_DOTS_ADDRESS = 142, /* RFC 8973 */
SD_DHCP6_OPTION_IPV6_ADDRESS_ANDSF = 143, /* RFC 6153 */
SD_DHCP6_OPTION_V6_DNR = 144 /* RFC 9463 */
/* option codes 145-65535 are unassigned */
SD_DHCP6_OPTION_IPV6_ADDRESS_ANDSF = 143, /* RFC 6153 */
SD_DHCP6_OPTION_V6_DNR = 144, /* RFC 9463 */
SD_DHCP6_OPTION_REGISTERED_DOMAIN = 145, /* RFC 9527 */
SD_DHCP6_OPTION_FORWARD_DIST_MANAGER = 146, /* RFC 9527 */
SD_DHCP6_OPTION_REVERSE_DIST_MANAGER = 147, /* RFC 9527 */
SD_DHCP6_OPTION_ADDR_REG_ENABLE = 148 /* RFC 9686 */
/* option codes 149-65535 are unassigned */
};
_SD_END_DECLARATIONS;

View File

@ -653,12 +653,12 @@ NotifyAccess=all
ExecStart=bash -x -c ' \
trap '"'"' \
now=\$\$(grep "^now" /proc/timer_list | cut -d" " -f3 | rev | cut -c 4- | rev); \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}\\n" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
(ls /etc | grep marker) >/tmp/markers/50i; \
(cat /usr/lib/os-release) >>/tmp/markers/50i; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
'"'"' SIGHUP; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
while true; do sleep 1; done; \
'
EOF
@ -688,12 +688,12 @@ NotifyAccess=all
ExecStart=bash -x -c ' \
trap '"'"' \
now=\$\$(grep "^now" /proc/timer_list | cut -d" " -f3 | rev | cut -c 4- | rev); \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}\\n" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
(ls /etc | grep marker) >/tmp/markers/50j; \
(cat /usr/lib/os-release) >>/tmp/markers/50j; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
'"'"' SIGHUP; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
while true; do sleep 1; done; \
'
EOF
@ -718,12 +718,12 @@ NotifyAccess=all
ExecStart=bash -x -c ' \
trap '"'"' \
now=\$\$(grep "^now" /proc/timer_list | cut -d" " -f3 | rev | cut -c 4- | rev); \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
printf "RELOADING=1\\nMONOTONIC_USEC=\$\${now}\\n" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
(ls /etc | grep marker) >/tmp/markers/50k; \
(cat /usr/lib/os-release) >>/tmp/markers/50k; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
'"'"' SIGHUP; \
echo -n "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
echo "READY=1" | socat -t 5 - UNIX-SENDTO:\$\$NOTIFY_SOCKET; \
while true; do sleep 1; done; \
'
EOF