MSI_dir_L1_MOSI_dir_L2_SNUCA_CMP
---------------------------------------


CMP System structure:

A CMP (Chip MultiProcessor) System is composed of one or more CMP chips.  A CMP chip consists of some number of processors, each with private L1 I+D caches and one shared L2 cache. The shared L2 cache can be sub-divided into banks where each bank as a separate cache controller.  One global interconnect is defined that connects all the components in the system including the L1 caches, the L2 cache banks, and the memories/directories.

High-level Protocol Description:

- The L1 has 3 stable states:  M, S, I

- L2 maintains strict inclusion and has full knowledge of on-chip L1 sharers via individual bits in the cache tag.  The stable
states are below:

   M,O,S,I :  normal meanings
        SS :  L2 entry shared, also present in one or more L1s or ext L2s
        SO :  L2 entry owned, also present in one or more L1s or ext L2s
        MT :  L2 entry modified in a local L1.  L2 copy is stale

- The protocol has a strict on-chip hierarchy where the L1 caches only communicate with their on-chip L2 cache, while off-chip the L2 cache communicates with the directory as well as directly responding to the other L2 caches.

High Level Event Description:

On a L1 GETS miss -
> The L2 controller will satisfy the miss immediately if in M, S, SS, or OS.
> If the L2 controller is in MT, the L2 controller will issue a down-grade request to the on-chip L1 sharer.  The L1 sharer will then downgrade to S and copy it's dirty block to the L2.  The L2 will then forward that copy to the L1 GETS requestor.
> If not present, the L2 will issue a GETS to the directory.  The directory will either forward the request, or respond with data.  If forwarded to another L2 controller, the L2 owner will respond with data directly to the L2 requestor which would then respond to the original L1 requestor.

On a L1 GETX miss -
> The L2 controller will satisfy the miss immediately if in M
> If the L2 controller is in MT, the L2 controller will issue an invalidate request to the on-chip L1 sharer.  The L1 sharer will then invalidate the block and writeback it's dirty block to the L2.  The L2 will then forward that copy to the L1 GETX requestor.
> If the L2 controller is in O, S, or I, the L2 controller will issue a GETX request to the directory.  The L2 will wait to receive the data and the necessary number of acks before responding to the L1 GETX requestor with the data and exclusive permission.
> If the L2 controller is in SS or SO, the L2 controller will issue a GETX request to the directory as well as invalidate requests to the on-chip L1 copies.  The L2 will wait to receive the data and the necessary number of acks (both on and off chip) before responding to the L1 GETX requestor with the data and exclusive permission.

Other minute details:

  The L2 acknowledges all L1 replacements
  The L1 acknowledges L2 invalidation requests
  The L2 sends an ack to the directory on a 3-hop transfer
  The forward request network (Dir -> L2 -> L1) is assumed to be pt-to-pt ordered
  The L1 request network (L1 -> L2) is assumed to be pt-to-pt ordered
  All other networks are unordered

Ruby Implementation Details:

- The new component network must be used with this protocol
- Each ruby node contains a L1 cache controller, 0 or more L2 cache controllers, and 0 or more directories.  There must be at least one directory in the system and at least one L2 cache controller per chip.  The directories are statically partitioned by address across the entire system (SMP), while the L2Cache banks are statically partitioned by address across a single chip.  There must be some power of 2 for each machine type.
- BUG ALERT: There is a single pool of TBE Entries for the L2 Controller.  Therefore it is possible to encounter a resource deadlock for these entries between satisfying L1 requests and directory requests.
