[scudo] Add a method to use a hard-coded page size (#106646)

Currently, only Android supports using a hard-code page size. Make this
a bit more generic so any platform that wants to can use this.

[scudo] Add a method to use a hard-coded page size (#106646)

Currently, only Android supports using a hard-code page size. Make this
a bit more generic so any platform that wants to can use this.

In addition, add a getPageSizeLogCached() function since this value is
used in release.h and can avoid keeping this value around in objects.

Finally, change some of the release.h page size multiplies to shifts
using the new page size log value.

show more ...

[scudo] Change isPowerOfTwo macro to return false for zero. (#87120)

Clean-up all of the calls and remove the redundant == 0 checks.

There is only one small visible change. For non-Android, the m

[scudo] Change isPowerOfTwo macro to return false for zero. (#87120)

Clean-up all of the calls and remove the redundant == 0 checks.

There is only one small visible change. For non-Android, the memalign
function will now fail if alignment is zero. Before this would have
passed.

show more ...

[scudo] Add utilization percentages for stats. (#75101)

Refactor the percentage display in the secondary code. Re-use that to
display a utilization percentage when displaying fragmentation data.

[scudo] Add specific die functions for linux specific failures. (#68650)

While running into failures on unmap calls, it becomes difficult to
figure out what's wrong. Break the dieOnMapUnmapError in

[scudo] Add specific die functions for linux specific failures. (#68650)

While running into failures on unmap calls, it becomes difficult to
figure out what's wrong. Break the dieOnMapUnmapError into specific
versions for map, unmap, and then one for mprotect.

Also, put these in a common linux space so that all linux derived code
can reuse this code.

show more ...

[scudo] Fine tune busy-waiting in HybridMutex

Instead of using hardware specific instruction, using simple loop over
volatile variable gives similar and more predicatable waiting time. Also
fine tun

[scudo] Fine tune busy-waiting in HybridMutex

Instead of using hardware specific instruction, using simple loop over
volatile variable gives similar and more predicatable waiting time. Also
fine tune the waiting time to fit with the average time in malloc/free
operations.

Reviewed By: cferris

Differential Revision: https://reviews.llvm.org/D156951

show more ...

[scudo] Remove RSS checking code.

The RSS code is not very useful and can be replicated by using
ulimit. Remove it and remove the options associated with it.

Reviewed By: Chia-hungDuan

Differentia

[scudo] Remove RSS checking code.

The RSS code is not very useful and can be replicated by using
ulimit. Remove it and remove the options associated with it.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D159155

show more ...

[scudo] Fix typo.

I copied this over incorrectly from my android tree. So fix the typo.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D158240

[scudo] Modify hard-coded page size for Android.

On Android, if PAGE_SIZE is defined, use that as the hard-coded value.
Otherwise, fallback to using getting the page size.

Reviewed By: Chia-hungDua

[scudo] Modify hard-coded page size for Android.

On Android, if PAGE_SIZE is defined, use that as the hard-coded value.
Otherwise, fallback to using getting the page size.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D158123

show more ...

[scudo] Add a method to force release everything.

The force flag to releaseToOSMaybe does not release everything
since it is an expensive operation. Modify the release flag to
have three states: nor

[scudo] Add a method to force release everything.

The force flag to releaseToOSMaybe does not release everything
since it is an expensive operation. Modify the release flag to
have three states: normal, force, forceall. Force behaves the same
as setting Force to true from before this change. Forceall will
release everything regardless of how much time it takes, or
how much there is to release.

In addition, add a new mallopt that will call the release function
with the forceall flag set.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D146106

show more ...

[scudo] Add a fast get time version.

On Android, the _COARSE version of clock_gettime is about twice as fast.
Therefore, add a getMonotonicTimeFast function that is used in the
releaseToOSMaybe func

[scudo] Add a fast get time version.

On Android, the _COARSE version of clock_gettime is about twice as fast.
Therefore, add a getMonotonicTimeFast function that is used in the
releaseToOSMaybe functions.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D145636

show more ...

[scudo] Improve the uses of roundUpTo/roundDownTo/isAligned

The implementations of those functions require the rounding target to be
power-of-two. It's better to add a debugging check to avoid misus

[scudo] Improve the uses of roundUpTo/roundDownTo/isAligned

The implementations of those functions require the rounding target to be
power-of-two. It's better to add a debugging check to avoid misuse.
Besides, add a general verion of those three to accommadate non
power-of-two cases.

Also change the name to roundUp/roundDown/isAligned

Reviewed By: cferris, cryptoad

Differential Revision: https://reviews.llvm.org/D142658

show more ...

[scudo] Fix return type of GetRSS()

[scudo][standalone] Precommit pages

On Fuchsia, this CL changes garbage collection
to precommit all pages if the |Buffer| doesn't
fit into the static buffer size.

A test program (scudotest) was use

[scudo][standalone] Precommit pages

On Fuchsia, this CL changes garbage collection
to precommit all pages if the |Buffer| doesn't
fit into the static buffer size.

A test program (scudotest) was used that deliberately
grows a size class high water mark to the point where
the pre-allocated static buffer is no longer used for
garbage collection.

Traces showed that precommiting the Vmar removes ~30 page faults
and ~.22ms of wall time.*

Before: https://ui.perfetto.dev/#!/?s=7da19fc3f59448eef51fd6fd03283bb87b702cf1a565bcbe6c9c28371671
After: https://ui.perfetto.dev/#!/?s=97707cd99b2c9efd1e6569b2deb97e3d16f8be532c59a0cc12463c37fbb1d8

*: Use the added `zx_vmar_op_range` as a reference point to observe
the differences.

For more context, see https://fxbug.dev/115594.

Reviewed By: Chia-hungDuan

Differential Revision: https://reviews.llvm.org/D140320

show more ...

[scudo] Reland: Fix and enable more warnings for standalone build

Differential Revision: https://reviews.llvm.org/D122761

scudo-standalone: Add GetRSS method on Linux

This change adds a GetRSS method on Linux that parses
the number from /proc/self/statm. This change is part
of splitting up https://reviews.llvm.org/D126

scudo-standalone: Add GetRSS method on Linux

This change adds a GetRSS method on Linux that parses
the number from /proc/self/statm. This change is part
of splitting up https://reviews.llvm.org/D126752.

Reviewed By: vitalybuka, cryptoad

Differential Revision: https://reviews.llvm.org/D139430

show more ...

[scudo] Add Scudo support for Trusty OS

trusty.cpp and trusty.h define Trusty implementations of map and other
platform-specific functions. In addition to adding Trusty configurations
in allocator_c

[scudo] Add Scudo support for Trusty OS

trusty.cpp and trusty.h define Trusty implementations of map and other
platform-specific functions. In addition to adding Trusty configurations
in allocator_config.h and size_class_map.h, MapSizeIncrement and
PrimaryEnableRandomOffset are added as configurable options in
allocator_config.h.
Background on Trusty: https://source.android.com/security/trusty

Differential Revision: https://reviews.llvm.org/D103578

show more ...

[scudo] Rework dieOnMapUnmapError

Said function had a few shortfalls:
- didn't set an abort message on Android
- was logged on several lines
- didn't provide extra information like the size requeste

[scudo] Rework dieOnMapUnmapError

Said function had a few shortfalls:
- didn't set an abort message on Android
- was logged on several lines
- didn't provide extra information like the size requested if OOM'ing

This improves the function to address those points.

Differential Revision: https://reviews.llvm.org/D103034

show more ...

scudo: Support memory tagging in the secondary allocator.

This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free

scudo: Support memory tagging in the secondary allocator.

This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free
and buffer underflow.

Use-after-free detection is implemented by setting memory page
protection to PROT_NONE on free. Because this must be done immediately
rather than after the memory has been quarantined, we no longer use the
combined allocator quarantine for secondary allocations. Instead, a
quarantine has been added to the secondary allocator cache.

Buffer overflow detection is implemented by aligning the allocation
to the right of the writable pages, so that any overflows will
spill into the guard page to the right of the allocation, which
will have PROT_NONE page protection. Because this would require the
secondary allocator to produce a header at the correct position,
the responsibility for ensuring chunk alignment has been moved to
the secondary allocator.

Buffer underflow detection has been implemented on hardware supporting
memory tagging by tagging the memory region between the start of the
mapping and the start of the allocation with a non-zero tag. Due to
the cost of pre-tagging secondary allocations and the memory bandwidth
cost of tagged accesses, the allocation itself uses a tag of 0 and
only the first four pages have memory tagging enabled.

This is a reland of commit 7a0da8894348 which was reverted in commit
9678b07e42ee. This reland includes the following changes:

- Fix the calculation of BlockSize which led to incorrect statistics
returned by mallinfo().
- Add -Wno-pedantic to silence GCC warning.
- Optionally add some slack at the end of secondary allocations to help
work around buggy applications that read off the end of their
allocation.

Differential Revision: https://reviews.llvm.org/D93731

show more ...

Revert 7a0da8894348, "scudo: Support memory tagging in the secondary allocator."

We measured a 2.5 seconds (17.5%) regression in Android boot time
performance with this change.

scudo: Support memory tagging in the secondary allocator.

This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free

scudo: Support memory tagging in the secondary allocator.

This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free
and buffer underflow.

Use-after-free detection is implemented by setting memory page
protection to PROT_NONE on free. Because this must be done immediately
rather than after the memory has been quarantined, we no longer use the
combined allocator quarantine for secondary allocations. Instead, a
quarantine has been added to the secondary allocator cache.

Buffer overflow detection is implemented by aligning the allocation
to the right of the writable pages, so that any overflows will
spill into the guard page to the right of the allocation, which
will have PROT_NONE page protection. Because this would require the
secondary allocator to produce a header at the correct position,
the responsibility for ensuring chunk alignment has been moved to
the secondary allocator.

Buffer underflow detection has been implemented on hardware supporting
memory tagging by tagging the memory region between the start of the
mapping and the start of the allocation with a non-zero tag. Due to
the cost of pre-tagging secondary allocations and the memory bandwidth
cost of tagged accesses, the allocation itself uses a tag of 0 and
only the first four pages have memory tagging enabled.

Differential Revision: https://reviews.llvm.org/D93731

show more ...

scudo: Add an API for disabling memory initialization per-thread.

Here "memory initialization" refers to zero- or pattern-init on
non-MTE hardware, or (where possible to avoid) memory tagging on MTE

scudo: Add an API for disabling memory initialization per-thread.

Here "memory initialization" refers to zero- or pattern-init on
non-MTE hardware, or (where possible to avoid) memory tagging on MTE
hardware. With shared TSD the per-thread memory initialization state
is stored in bit 0 of the TLS slot, similar to PointerIntPair in LLVM.

Differential Revision: https://reviews.llvm.org/D87739

show more ...

[scudo][standalone] mallopt runtime configuration options

Summary:
Partners have requested the ability to configure more parts of Scudo
at runtime, notably the Secondary cache options (maximum numbe

[scudo][standalone] mallopt runtime configuration options

Summary:
Partners have requested the ability to configure more parts of Scudo
at runtime, notably the Secondary cache options (maximum number of
blocks cached, maximum size) as well as the TSD registry options
(the maximum number of TSDs in use).

This CL adds a few more Scudo specific `mallopt` parameters that are
passed down to the various subcomponents of the Combined allocator.

- `M_CACHE_COUNT_MAX`: sets the maximum number of Secondary cached items
- `M_CACHE_SIZE_MAX`: sets the maximum size of a cacheable item in the Secondary
- `M_TSDS_COUNT_MAX`: sets the maximum number of TSDs that can be used (Shared Registry only)

Regarding the TSDs maximum count, this is a one way option, only
allowing to increase the count.

In order to allow for this, I rearranged the code to have some `setOption`
member function to the relevant classes, using the `scudo::Option` class
enum to determine what is to be set.

This also fixes an issue where a static variable (`Ready`) was used in
templated functions without being set back to `false` every time.

Reviewers: pcc, eugenis, hctim, cferris

Subscribers: jfb, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84667

show more ...

[scudo] Zero- and pattern-initialization of memory.

Summary:
Implement pattern initialization of memory (excluding the secondary
allocator because it already has predictable memory contents).
Expose

[scudo] Zero- and pattern-initialization of memory.

Summary:
Implement pattern initialization of memory (excluding the secondary
allocator because it already has predictable memory contents).
Expose both zero and pattern initialization through the C API.

Reviewers: pcc, cryptoad

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D79133

show more ...

scudo: Add support for diagnosing memory errors when memory tagging is enabled.

Introduce a function __scudo_get_error_info() that may be called to interpret
a crash resulting from a memory error, p

scudo: Add support for diagnosing memory errors when memory tagging is enabled.

Introduce a function __scudo_get_error_info() that may be called to interpret
a crash resulting from a memory error, potentially in another process,
given information extracted from the crashing process. The crash may be
interpreted as a use-after-free, buffer overflow or buffer underflow.

Also introduce a feature to optionally record a stack trace for each
allocation and deallocation. If this feature is enabled, a stack trace for
the allocation and (if applicable) the deallocation will also be available
via __scudo_get_error_info().

Differential Revision: https://reviews.llvm.org/D77283

show more ...

[scudo][standalone] Allow sched_getaffinity to fail

Summary:
In some configuration, `sched_getaffinity` can fail. Some reasons for
that being the lack of `CAP_SYS_NICE` capability or some syscall
fi

[scudo][standalone] Allow sched_getaffinity to fail

Summary:
In some configuration, `sched_getaffinity` can fail. Some reasons for
that being the lack of `CAP_SYS_NICE` capability or some syscall
filtering and so on.

This should not be fatal to the allocator, so in this situation, we
will fallback to the `MaxTSDCount` value specified in the allocator
configuration.

Reviewers: cferris, eugenis, hctim, morehouse, pcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D73055

show more ...