Features

• Supports DDR3 UDIMM/RDIMM/LRDIMM Interface on multiple platforms, e.g., Intel, AMD, Loongson

• Be able to collect full system off-chip memory traces with semantic information, including commercial applications

• Better signal integrity to support higher memory frequency signal acquisition

• Works at DDR3-800 properly, partly supports  DDR3-1600

• Supported memory size can be up to 512GB

• Uses PCI-Express cable to transport trace data, the peak bandwidth can achieve 8Gbps

• Uses 5-disk hardware SSD RAID, providing at about 2000MB/s IO write bandwidth and 5TB storage space for offline memory trace

• Cooperating with binary instrument software toolkit, we can further track memory traces with high level events

• Supports linux 3.10.93

• And more important: It is in use now!

Framework

• Memory Trace Board (MTB)

• Kernel Synchronization Module (KSMod)

• Trace packets capture, storage and analysis Toolkit

[framework]

Hardware Design

• Power supply

• DDR3 gold-finger

• DDR3 register

• 10 GE PHY

• PCI-Express external cable gen2 x4

• Xilinx Virtex-6 FPGA

• Multi-board synchronization module (optional)

The FPGA Physical Block Diagram

[fpga]

HMTT Hardware Board

[board]
HMTT3 illustrate

A working photo for collecting trace from a Intel Xeon E5 V2 Server

Software Design

Kernel Synchronization Module (KSMod)

• One module is added to alter the first page's cache attribution

• Another module is a more important module which provides the hmtt_printk routine to collect page table information and to store the information in a kernel-user shared buffer

• One kernel patch for pgtable.h records each update of page table

• Another kernel patch for entry.S sends identifiers to MTB when kernel-enter and kernel-exit occur (optional)

Trace packets capture and analysis Toolkit

• Memory Trace Board (MTB) Configuration

• Trace Dumping, Storage and Replay

• We had used HMTT3 to collect page table walk access ( All TLB miss access to page table.)