hwpmc: Restore line lost in previous commit

MFC after: 3 days

hwpmc: Fix whitespace in logging macros

MFC after: 1 week
Sponsored by: Klara, Inc.

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

pmc: Rework PROCEXEC event to support PIEs

Currently the PROCEXEC event only reports a single address, entryaddr,
which is the entry point of the interpreter in the typical dynamic case,
and used so

pmc: Rework PROCEXEC event to support PIEs

Currently the PROCEXEC event only reports a single address, entryaddr,
which is the entry point of the interpreter in the typical dynamic case,
and used solely to calculate the base address of the interpreter. For
PDEs this is fine, since the base address is known from the program
headers, but for PIEs the base address varies at run time based on where
the kernel chooses to load it, and so pmcstat has no way of knowing the
real address ranges for the executable. This was less of an issue in the
past since PIEs were rare, but now they're on by default on 64-bit
architectures it's more of a problem.

To solve this, pass through what was picked for et_dyn_addr by the
kernel, and use that as the offset for the executable's start address
just as is done for everything in the kernel. Since we're changing this
interface, sanitise the way we determine the interpreter's base address
by passing it through directly rather than indirectly via the entry
point and having to subtract off whatever the ELF header's e_entry is
(and anything that wants the entry point in future can still add that
back on as needed; this merely changes the interface to directly provide
the underlying variables involved).

This will be followed up by a bump to the pmc major version.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D39595

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

hwpmc_logging: less macro magic for type names

Provide the log type names in their entirely, rather than relying on the
macro to prepend the prefix. This improves their searchability; for
example, i

hwpmc_logging: less macro magic for type names

Provide the log type names in their entirely, rather than relying on the
macro to prepend the prefix. This improves their searchability; for
example, if I see PMCLOG_TYPE_PMCALLOCATE in libpmc I will now be able
to find where that is emitted in the kernel with a simple grep.

Reviewed by: jkoshy, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39911

show more ...

hwpmc: purge EOL release compatibility

Adjust pcmlog_{initialize,shutdown}() definitions to avoid clang 15 warning

With clang 15, the following -Werror warnings are produced:

sys/dev/hwpmc/hwpmc_logging.c:1228:18: error: a function

Adjust pcmlog_{initialize,shutdown}() definitions to avoid clang 15 warning

With clang 15, the following -Werror warnings are produced:

sys/dev/hwpmc/hwpmc_logging.c:1228:18: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
pmclog_initialize()
^
void
sys/dev/hwpmc/hwpmc_logging.c:1277:16: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
pmclog_shutdown()
^
void

This is because pcmlog_{initialize,shutdown}() are declared with (void)
argument lists, but defined with empty argument lists. Make the
definitions match the declarations.

MFC after: 3 days

show more ...

hwpmc: Increase thread priority while iterating CPUs.

This allows to profile already running high-priority threads, that
otherwise by blocking thread migration to respective CPUs blocked PMC
managem

hwpmc: Increase thread priority while iterating CPUs.

This allows to profile already running high-priority threads, that
otherwise by blocking thread migration to respective CPUs blocked PMC
management, i.e. profiling could start only when workload completed.

While there, return the thread to its original CPU after iterating
the list. Otherwise all threads using PMC end up on the last CPU.

MFC after: 1 month

show more ...

Update a sysctl name to nbuffers_pcpu in hwpmc.4 and pmcstat.c

This change was missed in r333509 (e6b475e0af).

Differential Revision: https://reviews.freebsd.org/D31704
Reviewed by: mjg

hwpmc: mostly clean up cc --analyze

Sponsored by: Rubicon Communications, LLC ("Netgate")

Merge ^/head r339813 through r340125.

Add malloc_domainset(9) and _domainset variants to other allocator KPIs.

Remove malloc_domain(9) and most other _domain KPIs added in r327900.
The new functions allow the caller to specify a general

Add malloc_domainset(9) and _domainset variants to other allocator KPIs.

Remove malloc_domain(9) and most other _domain KPIs added in r327900.
The new functions allow the caller to specify a general NUMA domain
selection policy, rather than specifically requesting an allocation from
a specific domain. The latter policy tends to interact poorly with
M_WAITOK, resulting in situations where a caller is blocked indefinitely
because the specified domain is depleted. Most existing consumers of
the _domain KPIs are converted to instead use a DOMAINSET_PREF() policy,
in which we fall back to other domains to satisfy the allocation
request.

This change also defines a set of DOMAINSET_FIXED() policies, which
only permit allocations from the specified domain.

Discussed with: gallatin, jeff
Reported and tested by: pho (previous version)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17418

show more ...

Merge ^/head r339015 through r339669.

MFH r338661 through r339200.

Sponsored by: The FreeBSD Foundation

hwpmc: Refactor sample ring buffer handling to fix races

Refactor sample ring buffer ring handling to make it more robust to
long running callchain collection handling

r338112 introduced a (now fix

hwpmc: Refactor sample ring buffer handling to fix races

Refactor sample ring buffer ring handling to make it more robust to
long running callchain collection handling

r338112 introduced a (now fixed) regression that exposed a number of race
conditions within the management of the sample buffers. This
simplifies the handling and moves the decision to overwrite a
callchain sample that has taken too long out of the NMI in to the
hardlock handler. With this change the problem no longer shows up as a
ring corruption but as the code spending all of its time in callchain
collection.

- Makes the producer / consumer index incrementing monotonic, making it
easier (for me at least) to reason about.
- Moves the decision to overwrite a sample from NMI context to interrupt
context where we can enforce serialization.
- Puts a time limit on waiting to collect a user callchain - putting a
bound on head-of-line blocking causing samples to be dropped
- Removes the flush routine which was previously needed to purge
dangling references to the pmc from the sample buffers but now is only
a source of a race condition on unload.

Previously one could lock up or crash HEAD by running:
pmcstat -S inst_retired.any_p -T and then hitting ^C

After this change it is no longer possible.

PR: 231793
Reviewed by: markj@
Approved by: re (gjb@)
Differential Revision: https://reviews.freebsd.org/D17011

show more ...

hwpmc: remove hacks to work around incorrect pc_domain

hwpmc: don't do EMIT64 on constant

pmc: convert native to jsonl and track TSC value of samples

- add '-j' options to filter to enable converting native pmc
log format to json lines format to enable the use of scripts
and external

pmc: convert native to jsonl and track TSC value of samples

- add '-j' options to filter to enable converting native pmc
log format to json lines format to enable the use of scripts
and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
recording nanotime when the sample is copied to a global log
in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
us an offset for translating the tsc values in callchain records

show more ...

hwpmc: add summary command and further metadata extensions

metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
to identify user vs kernel thread

hwpmc: add summary command and further metadata extensions

metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
idle: 138108207162
clang-6.0: 105336158004
sh: 72340108510
make: 8642012963
kernel: 7754011631
longest_lat_cache.miss:
clang-6.0: 87502625
sh: 40901227
make: 5500165
kernel: 3300099
awk: 2000060

% pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2

show more ...

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log

show more ...

hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
so that filter can identify which counter index an
event name maps to
- GC

hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
so that filter can identify which counter index an
event name maps to
- GC unused config flags
- make fixed counter assignment more robust as well as the
changes needed to be properly identified for filter

show more ...

hwpmc: support sampling both kernel and user stacks when interrupted in kernel

This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the syst

hwpmc: support sampling both kernel and user stacks when interrupted in kernel

This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.

Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.

Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks

show more ...

hwpmc: ensure that mapin updates are synchronous