Simone
88a8b7eb27
v2 said firmware filters CF8/CFC config writes to this function and that
the registers are reachable only through ECAM. That was never tested; the
only evidence was the boot line "PCI: Using configuration type 1", which
names the accessor and says nothing about a write failing.
Measured on the board, driver unloaded:
# setpci -s 16:1e.5 9c.L=20085002 # read SPD 0x50 register 0x02
# setpci -s 16:1e.5 9c.L
20005002 # GO consumed by the engine
# setpci -s 16:1e.5 a8.L
0000500c # READ_DONE, no error
# setpci -s 16:1e.5 b4.L
0000000c # SPD byte 2, DDR4 type code
The command executes and returns correct data through the ordinary
accessors, and MSR_SMI_COUNT does not move across it. So the driver now
uses pci_read_config_dword() and pci_write_config_dword(), the submission
is a single patch again, and nothing in it touches the PCI core. The
cover letter retracts the claim explicitly rather than dropping it
quietly, because v2 was posted with it.
Correctness work in the same pass:
- the completion bits are latched, so a command word that never reaches
the register is indistinguishable from a completed transfer and the
caller gets the previous transaction's data. Read the command word
back and require it to be the one just written; the engine only ever
clears GO, which the measurement above shows directly;
- compare the command registers after every transfer against what the
driver left in them and fail with -EAGAIN on a mismatch. The original
iMC driver had this check and it is what makes an unarbitrated system
detectable instead of silently wrong;
- use the i2c core's suspended-adapter mechanism rather than a private
flag and -EHOSTDOWN;
- use devm_mutex_init(), drop the unnecessary pci_enable_device(), and
return -ENODEV rather than -EPERM from the default-off gate, which was
printing a driver-core error on every boot;
- mask GO when restoring a saved command word.
An earlier version of the latched-DONE check required the GO bit to be
seen set or the status word to change. That was wrong: repeated transfers
with the same outcome leave STATUS bit for bit identical, and GO is
consumed faster than a config read-back can observe it. It produced false
positives on 3 of 15 smoke-test items before being replaced.
Add Documentation/i2c/busses/i2c-imc-skylake.rst, which every other PC
SMBus driver has and which was asked for on the 2020 iMC thread, and the
tools/ scripts for the hardware measurements: probe-cf8-write.sh and
probe-cf8-transaction.sh produced the result above, dump-pcu-config.sh is
the search for a TSOD polling control, watch-engine.sh is the firmware
activity measurement.
Smoke test on the X299 system with the current module: 15 checks, 0
failures, dmesg clean.
163 lines
8.0 KiB
Plaintext
163 lines
8.0 KiB
Plaintext
Subject: [PATCH v4] i2c: imc-skylake: add Intel Skylake-X iMC SMBus adapter
|
|
|
|
The integrated memory controller of Intel Skylake-X and Cascade Lake-X
|
|
processors contains an SMBus engine wired to the DDR4 DIMMs. This driver
|
|
exposes its two channels as standard Linux I2C adapters, so that DDR4 SPD
|
|
EEPROMs and DIMM thermal sensors can be read with i2c-tools, decode-dimms
|
|
and lm-sensors.
|
|
|
|
v2 was posted as i2c-imc-x299 and received no replies:
|
|
|
|
https://lore.kernel.org/linux-i2c/[email protected]/
|
|
|
|
v3 was never posted.
|
|
|
|
Correction to v2
|
|
~~~~~~~~~~~~~~~~
|
|
v2 claimed that firmware filters CF8/CFC config writes to this function and
|
|
that the registers can only be reached through the ECAM window. That claim
|
|
was wrong and I am retracting it.
|
|
|
|
It was never tested directly. The only evidence behind it was the boot log
|
|
line "PCI: Using configuration type 1", which says which accessor the kernel
|
|
uses and nothing about a write failing. Measured on the same board:
|
|
|
|
# setpci -s 16:1e.5 9c.L=20085002 # read SPD 0x50, register 0x02
|
|
# setpci -s 16:1e.5 9c.L
|
|
20005002 # GO consumed
|
|
# setpci -s 16:1e.5 a8.L
|
|
0000500c # READ_DONE, no error
|
|
# setpci -s 16:1e.5 b4.L
|
|
0000000c # SPD byte 2: DDR4 type code
|
|
|
|
The command written through CF8/CFC is executed and returns correct data,
|
|
and MSR_SMI_COUNT does not move across it. This driver therefore uses
|
|
pci_read_config_dword() and pci_write_config_dword() like any other driver.
|
|
There is no ECAM mapping, no MCFG parsing and no PCI-core change in this
|
|
series any more.
|
|
|
|
Firmware arbitration
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
This is the reason no iMC SMBus driver has ever been merged, and this patch
|
|
does not solve it either. It does make the condition detectable.
|
|
|
|
Andy Lutomirski's original i2c_imc put it plainly: a BMC, the memory
|
|
controller's power management, and possibly SMM all like to touch this bus,
|
|
and Intel defined no way to arbitrate between them. That driver already had
|
|
a module parameter called allow_unsafe_access. Stefan Schaeckeler's 2020
|
|
Broadwell rewrite went further and stopped the PCU's TSOD polling for the
|
|
duration of a transfer (write 0 to TSODCNTL, wait 10 ms for in-flight
|
|
transactions to drain, restore afterwards), because on Broadwell the
|
|
original driver's consistency check tripped about every 30 minutes.
|
|
|
|
https://lwn.net/Articles/635898/
|
|
https://lore.kernel.org/linux-i2c/[email protected]/
|
|
|
|
So this driver:
|
|
|
|
- keeps the default-off allow_unsafe_access gate, deliberately the same
|
|
name as the original;
|
|
- serializes both channels under one mutex and waits for the whole engine
|
|
to be idle before touching any register;
|
|
- saves the command state of both channels, clears TSOD_ACTIVE for the
|
|
duration of the transfer, and restores both afterwards;
|
|
- after every transfer compares the command registers against what it
|
|
left in them and fails the transfer with -EAGAIN if anything else
|
|
changed. This is the check the original driver had, and it is what
|
|
makes an unarbitrated system detectable rather than silently wrong;
|
|
- reads the command word back and requires it to be the one just
|
|
written, because the done bits are latched. With the driver unloaded
|
|
and nothing running, STATUS on the test system reads 0x0000300c:
|
|
READ_DONE and WRITE_DONE both still set from an earlier transaction.
|
|
Without the check, a command word that never reaches the register is
|
|
indistinguishable from a completed transfer and the caller receives the
|
|
previous transaction's data. The engine only ever clears GO, so that
|
|
is the single permitted difference between the two words.
|
|
|
|
What is missing: no equivalent of Broadwell's TSODCNTL has been identified
|
|
on this part. Offset 0xe0, where Broadwell has it, reads as zero here. A
|
|
system with CLTT enabled is therefore expected to trip the interference
|
|
check rather than be arbitrated against. I would rather ship a driver that
|
|
detects the condition and refuses than one that quietly races.
|
|
|
|
Relation to prior work
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
There is no iMC I2C driver in mainline. Lutomirski's i2c-imc was an RFC;
|
|
the 2020 resend and github.com/Linux4/i2c-imc are out of tree. Neither can
|
|
be extended to cover this part: the device ID (0x2085 vs 0x3ca8/0x6fa8) and
|
|
the register layout (0x9C-0xB8 vs 0x180-0x188) differ. A separate file is
|
|
cleaner than a hw_data table.
|
|
|
|
The register encoding was cross-checked against an independent Windows
|
|
implementation for the same engine, which predates this work:
|
|
|
|
https://github.com/namazso/PawnIO.Modules/blob/c975ca1d1b8a9c7473a175fa1ca24d83b6cbc8f3/SmbusIntelSkylakeIMC.p
|
|
|
|
It also implements SMBus BYTE through a pointer-select bit. That is not
|
|
advertised here because it has not been validated on the Linux system, so
|
|
i2cdetect cannot scan these buses; devices are addressed directly.
|
|
|
|
Changes since v2
|
|
~~~~~~~~~~~~~~~~
|
|
- Retract the CF8/CFC claim; use the ordinary config accessors. The
|
|
driver no longer maps anything.
|
|
- Rename i2c-imc-x299 to i2c-imc-skylake; the engine is in the CPU, not
|
|
the chipset.
|
|
- Add the default-off allow_unsafe_access gate.
|
|
- Wait for both channels to be idle before touching the data latch.
|
|
- Preserve and restore command and TSOD state for both channels.
|
|
- Validate READ_DONE, WRITE_DONE and the hardware error state.
|
|
- Detect interference from another master and report it as -EAGAIN.
|
|
- Read the command word back and require it to be the one written.
|
|
- Add bounded command-toggle recovery for a stuck BUSY state.
|
|
- Add WORD_DATA support with the hardware byte ordering.
|
|
- Reject I2C_M_TEN explicitly and validate transfer direction.
|
|
- Use the i2c core's suspended-adapter mechanism instead of a private
|
|
flag and a non-standard errno.
|
|
- Use devm_mutex_init() and devm-managed adapters; drop the unnecessary
|
|
pci_enable_device(); return -ENODEV rather than -EPERM from the
|
|
default-off gate, which was logging a driver-core error on every boot.
|
|
- Add Documentation/i2c/busses/i2c-imc-skylake.rst.
|
|
- Drop the ACPI dependency.
|
|
|
|
Validation
|
|
~~~~~~~~~~
|
|
- In-tree build, W=1, x86_64 GCC and arm64 Clang COMPILE_TEST: no
|
|
diagnostics.
|
|
- checkpatch.pl --strict: clean.
|
|
- Sparse with __CHECK_ENDIAN__: clean.
|
|
- Read-only hardware test on one Intel Core i9-10900X (Cascade Lake-X)
|
|
system with four DDR4 DIMMs: both adapters, four SPD EEPROMs, BYTE_DATA,
|
|
WORD_DATA byte order, ENXIO for an absent device, 100 concurrent reads
|
|
per channel, the default-off gate, and 20 unload/reload cycles: 15
|
|
checks, 0 failures, no kernel warning or oops, no interference
|
|
reported, every command word confirmed in the register.
|
|
- Config space of the PCU function sampled with the driver unloaded: no
|
|
register changed on its own, and TSOD_ACTIVE was clear on both channels,
|
|
so no firmware activity was observed on this system.
|
|
|
|
Still missing: an independent Tested-by, ideally on Skylake-X rather than
|
|
Cascade Lake-X, and suspend/resume testing with active I2C clients.
|
|
|
|
Tool assistance
|
|
~~~~~~~~~~~~~~~
|
|
Per Documentation/process/generated-content.rst: this work was developed
|
|
with an AI coding assistant, run interactively rather than autonomously.
|
|
|
|
- What: Claude Code, model claude-opus-5.
|
|
- Where: it contributed to the transaction state machine, the .rst
|
|
documentation, the commit message and this cover letter. The register
|
|
decoding is mine.
|
|
- How it was used: iteratively, one reviewed change at a time, in
|
|
response to specific requests, not from a single generating prompt. A
|
|
representative request is "the done bits are latched; show me how a
|
|
command that is never accepted can be mistaken for a completed one, and
|
|
make that case detectable", which produced imc_engine_responded(). It
|
|
also designed the measurement that retracted the CF8/CFC claim above,
|
|
after asking why that claim had never been tested directly.
|
|
- Testing: every revision was built with W=1, checked with
|
|
checkpatch --strict and sparse, and tested on the hardware by me.
|
|
|
|
I have reviewed all of it, I understand it, and I take responsibility for
|
|
it.
|