Serialize rdtsc using with lfence, mfence or cpuid to read TSC more precisely.

x86/x86/tsc.c rev. 1.67 reduced cache problem and got big improvement, but it
still has room. I measured the effect o

Serialize rdtsc using with lfence, mfence or cpuid to read TSC more precisely.

x86/x86/tsc.c rev. 1.67 reduced cache problem and got big improvement, but it
still has room. I measured the effect of lfence, mfence, cpuid and rdtscp.
The impact to TSC skew and/or drift is:

AMD: mfence > rdtscp > cpuid > lfence-serialize > lfence = nomodify
Intel: lfence > rdtscp > cpuid > nomodify

So, mfence is the best on AMD and lfence is the best on Intel. If it has no
SSE2, we can use cpuid.

NOTE:
- An AMD's document says DE_CFG_LFENCE_SERIALIZE bit can be used for
serializing, but it's not so good.
- On Intel i386(not amd64), it seems the improvement is very little.
- rdtscp instruct can be used as serializing instruction + rdtsc, but
it's not good as [lm]fence. Both Intel and AMD's document say that
the latency of rdtscp is bigger than rdtsc, so I suspect the difference
of the result comes from it.

show more ...

Fix the TSC timecounter (on the systems I have access to):

- Make the early i8254-based calculation of frequency a bit more accurate.

- Keep track of how far the HPET & TSC advance between HPET att

Fix the TSC timecounter (on the systems I have access to):

- Make the early i8254-based calculation of frequency a bit more accurate.

- Keep track of how far the HPET & TSC advance between HPET attach and
secondary CPU boot, and use to compute an accurate value before attaching
the timecounter. Initial idea from joerg@.

- When determining skew and drift between CPUs, make each measurement 1000
times and pick the lowest observed value. Increase the error threshold to
1000 clock cycles.

- Use the frequency computed on the boot CPU for secondary CPUs too.

- Remove cpu_counter_serializing().

show more ...

Some CPU have cpu counter (CPUID_TSC is there) but don't handle the
rdmsr instruction (CPUID_MSR is not there).
Introduce a cpu_counter_serializing() function to remplace rdmsr(MSR_TSC)
calls, which

Some CPU have cpu counter (CPUID_TSC is there) but don't handle the
rdmsr instruction (CPUID_MSR is not there).
Introduce a cpu_counter_serializing() function to remplace rdmsr(MSR_TSC)
calls, which does a rdmsr(MSR_TSC) if available and cpu_counter() otherwise.
This makes the cpu counter useable on vortex86 CPUs.
OK ad@

show more ...

Improve x86 tsc handling:

- Ditch the cross-CPU calibration stuff. It didn't work properly, and it's
near impossible to synchronize the CPUs in a running system, because bus
traffic will interfe

Improve x86 tsc handling:

- Ditch the cross-CPU calibration stuff. It didn't work properly, and it's
near impossible to synchronize the CPUs in a running system, because bus
traffic will interfere with any calibration attempt, messing up the
timings.

- Only enable the TSC on CPUs where we are sure it does not drift. If we are
On a known good CPU, give the TSC high timecounter quality, making it the
default.

- When booting CPUs, detect TSC skew and account for it. Most Intel MP
systems have synchronized counters, but that need not be true if the
system has a complicated bus structure. As far as I know, AMD systems
do not have synchronized TSCs and so we need to handle skew.

- While an AP is waiting to be set running, try and make the TSC drift by
entering a reduced power state. If we detect drift, ensure that the TSC
does not get a high timecounter quality. This should not happen and is
only for safety.

- Make cpu_counter() stuff LKM safe.

show more ...

Merge cpu_counter.h.

Remove clause 3 and 4 from TNF licenses

Move x86 common cpu_counter functions into <x86/cpu_counter.h>.