Lines Matching full:shadow
15 /// We associate a few shadow bits with every byte of the application memory,
16 /// poison the shadow of the malloc-ed or alloca-ed memory, load the shadow,
17 /// bits on every memory read, propagate the shadow bits through some of the
18 /// arithmetic instruction (including MOV), store the shadow bits on every memory
20 /// associated shadow is poisoned.
30 /// Another difference from Memcheck is that we use 8 shadow bits per
31 /// byte of application memory and use a direct shadow mapping. This
33 /// shadow updates (Memcheck is single-threaded so races are not a
34 /// concern there. Memcheck uses 2 shadow bits per byte with a slow
37 /// The default value of shadow is 0, which means "clean" (not poisoned).
40 /// shadow memory is ready. On error, __msan_warning is called. Since
42 /// specialized thread-local shadow for return values
52 /// They are stored in a second shadow mapping, one 4-byte value for 4 bytes
71 /// corresponding shadow location in an atomic way. Unfortunately, atomic store
77 /// not the other way around. We load the shadow _after_ the application load,
78 /// and we store the shadow _before_ the app store. Also, we always store clean
79 /// shadow (if the application store is atomic). This way, if the store-load
80 /// pair constitutes a happens-before arc, shadow store and load are correctly
86 /// must store the new shadow before the app operation, and load the shadow
90 /// clean shadow.
104 /// it's safe to touch the shadow memory.
106 /// the assembly call, so that changes to the shadow memory will be seen by
113 /// - KMSAN allocates shadow and origin memory for each page separately, so
114 /// there are no explicit accesses to shadow and origin in the
116 /// Shadow and origin values for a particular X-byte memory location
121 /// are possible and returns the pointers to shadow and origin memory.
125 /// Note that the sanitizer code has to deal with how shadow/origin pairs
272 cl::desc("propagate shadow through ICmpEQ and ICmpNE"),
299 // This flag controls whether we check the shadow of the address
307 cl::desc("report accesses through a pointer which has poisoned shadow"),
339 ClCheckConstantShadow("msan-check-constant-shadow",
340 cl::desc("Insert checks for constant shadow values"),
360 static cl::opt<uint64_t> ClShadowBase("msan-shadow-base",
379 // Memory map parameters used in application-to-shadow address calculation.
381 // Shadow = ShadowBase + Offset
591 /// Thread-local shadow storage for function parameters.
597 /// Thread-local shadow storage for function return value.
603 /// Thread-local shadow storage for in-register va_arg function.
606 /// Thread-local shadow storage for in-register va_arg function.
609 /// Thread-local shadow storage for va_arg overflow area.
641 /// KMSAN callback for task-local function argument shadow.
648 /// Pair of shadow/origin pointers.
663 /// Memory map parameters used in application-to-shadow calculation.
666 /// Custom memory map parameters used when -msan-shadow-base or
767 // SystemZ ABI: shadow/origin pair is returned via a hidden parameter.
1076 /// necessary to propagate argument shadow through VarArg function
1128 /// instructions store and load corresponding shadow and origin
1129 /// values. Most instructions propagate shadow from arguments to their
1131 /// test their argument shadow and print reports (with a runtime call) if it's
1151 Value *Shadow;
1156 : Shadow(S), Origin(O), OrigIns(I) {}
1178 // blocks, such nodes will not have any shadow value associated with them.
1179 // It's easier to remove unreachable blocks than deal with missing shadow.
1282 void storeOrigin(IRBuilder<> &IRB, Value *Addr, Value *Shadow, Value *Origin,
1286 TypeSize StoreSize = DL.getTypeStoreSize(Shadow->getType());
1288 Value *ConvertedShadow = convertShadowToScalar(Shadow, IRB);
1291 // Origin is not needed: value is initialized or const shadow is
1329 Value *Shadow = SI->isAtomic() ? getCleanShadow(Val) : getShadow(Val);
1331 Type *ShadowTy = Shadow->getType();
1337 StoreInst *NewSI = IRB.CreateAlignedStore(Shadow, ShadowPtr, Alignment);
1345 storeOrigin(IRB, Addr, Shadow, getOrigin(Val), OriginPtr,
1427 // Disable combining in some cases. TrackOrigins checks each shadow to pick
1431 Value *Shadow = nullptr;
1436 Value *ConvertedShadow = ShadowData.Shadow;
1440 // Skip, value is initialized or const shadow is ignored.
1459 if (!Shadow) {
1460 Shadow = ConvertedShadow;
1464 Shadow = convertToBool(Shadow, IRB, "_mscmp");
1466 Shadow = IRB.CreateOr(Shadow, ConvertedShadow, "_msor");
1469 if (Shadow) {
1472 materializeOneCheck(IRB, Shadow, nullptr);
1526 // Iterate all BBs in depth-first order and create shadow instructions
1528 // For PHI nodes we create dummy shadow PHIs which will be finalized later.
1564 // Insert shadow value checks.
1574 /// Compute the shadow type that corresponds to a given Value.
1577 /// Compute the shadow type that corresponds to a given Type.
1582 // For integer type, shadow is the same as the original type.
1608 /// Extract combined shadow of struct elements as a bool
1609 Value *collapseStructShadow(StructType *Struct, Value *Shadow,
1615 // Combine by ORing together each element's bool shadow
1616 Value *ShadowItem = IRB.CreateExtractValue(Shadow, Idx);
1628 // Extract combined shadow of array elements
1629 Value *collapseArrayShadow(ArrayType *Array, Value *Shadow,
1634 Value *FirstItem = IRB.CreateExtractValue(Shadow, 0);
1638 Value *ShadowItem = IRB.CreateExtractValue(Shadow, Idx);
1645 /// Convert a shadow value to it's flattened variant. The resulting
1646 /// shadow may not necessarily have the same bit width as the input
1702 /// Compute the integer shadow offset that corresponds to a given
1706 /// Addr can be a ptr or <N x ptr>. In both cases ShadowTy the shadow type of
1721 /// Compute the shadow and origin addresses corresponding to a given
1724 /// Shadow = ShadowBase + Offset
1726 /// Addr can be a ptr or <N x ptr>. In both cases ShadowTy the shadow type of
1804 /// Addr can be a ptr or <N x ptr>. In both cases ShadowTy the shadow type of
1849 /// Compute the shadow address for a given function argument.
1851 /// Shadow = ParamTLS+ArgOffset.
1869 /// Compute the shadow address for a retval.
1880 /// Set SV to be the shadow value for V.
1882 assert(!ShadowMap.count(V) && "Values may only have one shadow");
1902 /// Create a clean shadow value for a given value.
1904 /// Clean shadow (all zeroes) means all bits of the value are defined
1908 /// Create a dirty shadow of a given shadow type.
1924 llvm_unreachable("Unexpected shadow type");
1927 /// Create a dirty shadow for a given value.
1938 /// Get the shadow value for a given Value.
1946 // For instructions the shadow is already stored in the map.
1947 Value *Shadow = ShadowMap[V];
1948 if (!Shadow) {
1949 LLVM_DEBUG(dbgs() << "No shadow: " << *V << "\n" << *(I->getParent()));
1951 assert(Shadow && "No shadow for a value");
1953 return Shadow;
1963 // For arguments we compute the shadow on demand and store it in the map.
1991 // ByVal pointer itself has clean shadow. We copy the actual
1992 // argument shadow to the underlying memory.
2033 // Shadow over TLS
2050 assert(ShadowPtr && "Could not find shadow for an argument");
2053 // For everything else the shadow is zero.
2057 /// Get the shadow for i-th argument of the instruction I.
2084 /// Remember the place where a shadow check should be inserted.
2087 /// UMR warning in runtime if the shadow value is not 0.
2088 void insertShadowCheck(Value *Shadow, Value *Origin, Instruction *OrigIns) {
2089 assert(Shadow);
2094 LLVM_DEBUG(dbgs() << "Skipping check of " << *Shadow << " before "
2099 Type *ShadowTy = Shadow->getType();
2102 "Can only insert checks for integer, vector, and aggregate shadow "
2106 ShadowOriginAndInsertPoint(Shadow, Origin, OrigIns));
2109 /// Remember the place where a shadow check should be inserted.
2115 Value *Shadow, *Origin;
2117 Shadow = getShadow(Val);
2118 if (!Shadow)
2122 Shadow = dyn_cast_or_null<Instruction>(getShadow(Val));
2123 if (!Shadow)
2127 insertShadowCheck(Shadow, Origin, OrigIns);
2208 // We still need to set the shadow and origin to clean values.
2219 /// Loads the corresponding shadow and (optionally) origin.
2257 /// Stores the corresponding shadow and (optionally) origin.
2380 /// Propagate shadow for bitwise AND.
2384 /// corresponding bit in B don't affect the resulting shadow.
2429 /// Default propagation of shadow and/or origin.
2431 /// This class implements the general case of shadow propagation, used in all
2433 /// actually does. It converts all input shadow values to a common type
2445 Value *Shadow = nullptr;
2454 /// Add a pair of shadow and origin values to the mix.
2458 if (!Shadow)
2459 Shadow = OpShadow;
2461 OpShadow = MSV->CreateShadowCast(IRB, OpShadow, Shadow->getType());
2462 Shadow = IRB.CreateOr(Shadow, OpShadow, "_msprop");
2489 /// Set the current combined values as the given instruction's shadow
2493 assert(Shadow);
2494 Shadow = MSV->CreateShadowCast(IRB, Shadow, MSV->getShadowTy(I));
2495 MSV->setShadow(I, Shadow);
2529 "Vector of pointers is not a valid shadow type");
2535 /// Cast between two shadow types, extending or truncating as
2560 /// Cast an application value to the type of its own shadow.
2571 /// Propagate shadow for arbitrary operation.
2587 // shadow left by the required number of bits. Effectively, we transform
2658 // divisor is fully initialized and must propagate shadow. See PR37523.
2705 // Split shadow into sign bit and other bits.
2708 // Maximise the undefined shadow bit, minimize other undefined bits.
2722 // Split shadow into sign bit and other bits.
2725 // Minimise the undefined shadow bit, maximise other undefined bits.
2736 /// This function does exact shadow propagation for all relational
2770 /// bit of the shadow. Everything else is delegated to handleShadowOr().
2791 Value *Shadow = IRB.CreateICmpSLT(getShadow(op), getCleanShadow(op),
2793 setShadow(&I, Shadow);
2872 /// we will memove the shadow twice: which is bad in case
2877 getShadow(I.getArgOperand(1)); // Ensure shadow initialized
2887 /// Similar to memmove: avoid copying shadow twice. This is somewhat
2900 getShadow(I.getArgOperand(1)); // Ensure shadow initialized
2930 Value *Shadow = getShadow(&I, 1);
2936 Addr, IRB, Shadow->getType(), Align(1), /*isStore*/ true);
2937 IRB.CreateAlignedStore(Shadow, ShadowPtr, Align(1));
3074 // Set the Output shadow based on input Shadow
3077 // If zero poison is requested, mix in with the shadow
3103 // We copy the shadow of \p CopyOp[NumUsedElements:] to \p
3131 // Combine shadow for elements of ConvertOp that are used in this operation,
3133 // FIXME: consider propagating shadow of ConvertOp, at least in the case of
3151 // Build result shadow by zero-filling parts of CopyOp shadow that come from
3265 // Shadow is propagated with the signed variant of the same intrinsic applied
3315 // Calculate output shadow as array of booleans `<n x i1>`, assuming if any
3358 // Calculate shadow as `<n x i1>`.
3361 // First 4 elements of shadow are already calculated. `makeDppShadow`
3366 // Extend to real size of shadow, poisoning either all or none bits of an
3394 // Extract top bit from condition and its shadow.
3445 // all-ones shadow.
3482 // corresponding shadow bits.
3500 // corresponding shadow bits.
3543 Value *Shadow = IRB.CreateAlignedLoad(Ty, ShadowPtr, Alignment, "_ldmxcsr");
3546 insertShadowCheck(Shadow, Origin, &I);
3571 Value *Shadow = IRB.CreateMaskedExpandLoad(
3574 setShadow(&I, Shadow);
3591 Value *Shadow = getShadow(Values);
3597 IRB.CreateMaskedCompressStore(Shadow, ShadowPtr, Mask);
3630 Value *Shadow =
3634 setShadow(&I, Shadow);
3657 Value *Shadow = getShadow(Values);
3663 IRB.CreateMaskedScatter(Shadow, ShadowPtrs, Alignment, Mask);
3675 Value *Shadow = getShadow(V);
3685 Ptr, IRB, Shadow->getType(), Alignment, /*isStore*/ true);
3687 IRB.CreateMaskedStore(Shadow, ShadowPtr, Alignment, Mask);
3694 DL.getTypeStoreSize(Shadow->getType()),
3752 // Apply the same intrinsic to the shadow of the first operand.
3775 // and then apply the usual shadow combining logic.
3805 Value *Shadow = IRB.CreateShuffleVector(First, Second, Mask);
3807 setShadow(&I, Shadow);
3818 Value *Shadow = IRB.CreateZExt(Scalar, getShadowTy(&I));
3820 setShadow(&I, Shadow);
3836 Value *Shadow = IRB.CreateShuffleVector(First, OrShadow, Mask);
3838 setShadow(&I, Shadow);
3857 Value *Shadow = getShadow(&I, 0);
3858 setShadow(&I, IRB.CreateICmpNE(Shadow, getCleanShadow(Shadow)));
3870 Value *Shadow = PoisonValue::get(getShadowTy(&I));
3871 Shadow = IRB.CreateInsertValue(Shadow, ShadowElt0, 0);
3872 Shadow = IRB.CreateInsertValue(Shadow, ShadowElt1, 1);
3874 setShadow(&I, Shadow);
3904 // MSan's GetShadowTy assumes the LHS is the type we want the shadow for
3907 // we know the type of the output (and its shadow) is <16 x i8>.
4304 // the shadow operations aren't reordered before it.
4334 // the shadow operations aren't reordered after it.
4344 // Atomic store always paints clean shadow/origin. See file header.
4353 // do the usual thing: check argument shadow and mark all outputs as
4437 // Compute the Shadow for arg even if it is ByVal, because
4438 // in that case getShadow() will copy the actual arg shadow to
4443 << " Shadow: " << *ArgShadow << "\n");
4507 // Now, get the shadow for the RetVal.
4521 // Until we have full dynamic coverage, make sure the retval shadow is 0.
4543 "Could not find insertion point for retval shadow load");
4580 Value *Shadow = getShadow(RetVal);
4584 Shadow = getCleanShadow(RetVal);
4591 IRB.CreateAlignedStore(Shadow, ShadowPtr, kShadowTLSAlignment);
4703 // Result shadow if condition shadow is 0.
4717 // Cast arguments to shadow-compatible type.
4721 // Result shadow if condition shadow is 1.
4829 // the pointer. Conservatively assume that the shadow memory is unaligned.
4869 // Conservative inline assembly handling: check for poisoned shadow of
4900 // call, so that the shadow for memory published in the asm() statement
4919 // Everything else: stop propagating and check for poisoned shadow.
4949 /// Compute the shadow address for a given va_arg.
4958 /// Compute the shadow address for a given va_arg.
4981 // value shadow, but it will be copied to backup anyway. Make it
5060 // For VarArg functions, store the argument shadow in an ABI-specific format
5066 // MSanParamTLS, and extract shadow on va_arg() call in the argument list
5097 continue; // We have no space to copy shadow there.
5144 // We have no space to copy shadow there.
5154 Value *Shadow = MSV.getShadow(A);
5155 IRB.CreateAlignedStore(Shadow, ShadowBase, kShadowTLSAlignment);
5158 TypeSize StoreSize = DL.getTypeStoreSize(Shadow->getType());
5199 // Copy va_list shadow from the backup copy of the TLS contents.
5265 // Adjusting the shadow for argument with size < 8 to match the
5308 // Copy va_list shadow from the backup copy of the TLS contents.
5374 // The instrumentation stores the argument shadow in a non ABI-specific
5419 // We have no space to copy shadow there.
5426 // bother to actually store a shadow.
5481 // Instrument va_start, copy va_list shadow from the backup copy of
5491 // We need then to propagate the shadow arguments on both regions
5493 // The remaining arguments are saved on shadow for 'va::stack'.
5496 // saved. So to copy the shadow values from the va_arg TLS array
5523 // argument by ignoring the bytes of shadow from named arguments.
5642 // Adjusting the shadow for argument with size < 8 to match the
5692 // Copy va_list shadow from the backup copy of the TLS contents.
5768 // extension". Shadow for an integer argument has the same type as the
5810 // Always keep track of GpOffset, but store shadow only for varargs.
5832 // Always keep track of FpOffset, but store shadow only for varargs.
5839 // don't extend shadow and don't mind the gap.
5851 // Keep track of VrIndex. No need to store shadow, since vector varargs
5858 // Keep track of OverflowOffset and store shadow only for varargs.
5860 // the overflow area shadow.
5885 Value *Shadow = MSV.getShadow(A);
5887 Shadow = MSV.CreateShadowCast(IRB, Shadow, IRB.getInt64Ty(),
5890 ShadowBase, PointerType::get(Shadow->getType(), 0), "_msarg_va_s");
5891 IRB.CreateStore(Shadow, ShadowBase);
5894 TypeSize StoreSize = DL.getTypeStoreSize(Shadow->getType());
5930 // don't know real overflow size and can't clear shadow beyond kParamTLSSize.
5988 // Copy va_list shadow from the backup copy of the TLS contents.