279 lines
12 KiB
Plaintext
279 lines
12 KiB
Plaintext
|
# Copyright (c) 2012 ARM Limited
|
|||
|
# All rights reserved
|
|||
|
#
|
|||
|
# The license below extends only to copyright in the software and shall
|
|||
|
# not be construed as granting a license to any other intellectual
|
|||
|
# property including but not limited to intellectual property relating
|
|||
|
# to a hardware implementation of the functionality of the software
|
|||
|
# licensed hereunder. You may use the software subject to the license
|
|||
|
# terms below provided that you ensure that this notice is replicated
|
|||
|
# unmodified and in its entirety in all distributions of the software,
|
|||
|
# modified or unmodified, in source code or in binary form.
|
|||
|
#
|
|||
|
# Redistribution and use in source and binary forms, with or without
|
|||
|
# modification, are permitted provided that the following conditions are
|
|||
|
# met: redistributions of source code must retain the above copyright
|
|||
|
# notice, this list of conditions and the following disclaimer;
|
|||
|
# redistributions in binary form must reproduce the above copyright
|
|||
|
# notice, this list of conditions and the following disclaimer in the
|
|||
|
# documentation and/or other materials provided with the distribution;
|
|||
|
# neither the name of the copyright holders nor the names of its
|
|||
|
# contributors may be used to endorse or promote products derived from
|
|||
|
# this software without specific prior written permission.
|
|||
|
#
|
|||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
|||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
|||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
|||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
|||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
|||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
|||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
|||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
|||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|||
|
#
|
|||
|
# Author: Djordje Kovacevic
|
|||
|
|
|||
|
/*! \page gem5MemorySystem Memory System in gem5
|
|||
|
|
|||
|
\tableofcontents
|
|||
|
|
|||
|
The document describes memory subsystem in gem5 with focus on program flow
|
|||
|
during CPU’s simple memory transactions (read or write).
|
|||
|
|
|||
|
|
|||
|
\section gem5_MS_MH MODEL HIERARCHY
|
|||
|
|
|||
|
Model that is used in this document consists of two out-of-order (O3)
|
|||
|
ARM v7 CPUs with corresponding L1 data caches and Simple Memory. It is
|
|||
|
created by running gem5 with the following parameters:
|
|||
|
|
|||
|
configs/example/fs.py --caches --cpu-type=arm_detailed --num-cpus=2
|
|||
|
|
|||
|
Gem5 uses Memory Objects (MemObject) derived objects as basic blocks for
|
|||
|
building memory system. They are connected via ports with established
|
|||
|
master/slave hierarchy. Data flow is initiated on master port while the
|
|||
|
response messages and snoop queries appear on the slave port. The following
|
|||
|
figure shows the hierarchy of Memory Objects used in this document:
|
|||
|
|
|||
|
\image html "gem5_MS_Fig1.PNG" "Memory Object hierarchy of the model" width=3cm
|
|||
|
|
|||
|
\section gem5_CPU CPU
|
|||
|
|
|||
|
It is not in the scope of this document to describe O3 CPU model in details, so
|
|||
|
here are only a few relevant notes about the model:
|
|||
|
|
|||
|
<b>Read access </b>is initiated by sending message to the port towards DCache
|
|||
|
object. If DCache rejects the message (for being blocked or busy) CPU will
|
|||
|
flush the pipeline and the access will be re-attempted later on. The access
|
|||
|
is completed upon receiving reply message (ReadRep) from DCache.
|
|||
|
|
|||
|
<b>Write access</b> is initiated by storing the request into store buffer whose
|
|||
|
context is emptied and sent to DCache on every tick. DCache may also reject
|
|||
|
the request. Write access is completed when write reply (WriteRep) message is
|
|||
|
received from DCache.
|
|||
|
|
|||
|
Load & store buffers (for read and write access) don’t impose any
|
|||
|
restriction on the number of active memory accesses. Therefore, the maximum
|
|||
|
number of outstanding CPU’s memory access requests is not limited by CPU
|
|||
|
Memory Object but by underlying memory system model.
|
|||
|
|
|||
|
<b>Split memory access</b> is implemented.
|
|||
|
|
|||
|
The message that is sent by CPU contains memory type (Normal, Device, Strongly
|
|||
|
Ordered and cachebility) of the accessed region. However, this is not being used
|
|||
|
by the rest of the model that takes more simplified approach towards memory types.
|
|||
|
|
|||
|
\section gem5_DCache DATA CACHE OBJECT
|
|||
|
|
|||
|
Data Cache object implements a standard cache structure:
|
|||
|
|
|||
|
\image html "gem5_MS_Fig2.PNG" "DCache Memory Object" width=3cm
|
|||
|
|
|||
|
<b>Cached memory reads</b> that match particular cache tag (with Valid & Read
|
|||
|
flags) will be completed (by sending ReadResp to CPU) after a configurable time.
|
|||
|
Otherwise, the request is forwarded to Miss Status and Handling Register
|
|||
|
(MSHR) block.
|
|||
|
|
|||
|
<b>Cached memory writes</b> that match particular cache tag (with Valid, Read
|
|||
|
& Write flags) will be completed (by sending WriteResp CPU) after the same
|
|||
|
configurable time. Otherwise, the request is forwarded to Miss Status and
|
|||
|
Handling Register(MSHR) block.
|
|||
|
|
|||
|
<b>Uncached memory reads</b> are forwarded to MSHR block.
|
|||
|
|
|||
|
<b>Uncached memory writes</b> are forwarded to WriteBuffer block.
|
|||
|
|
|||
|
<b>Evicted (& dirty) cache lines</b> are forwarded to WriteBuffer block.
|
|||
|
|
|||
|
CPU’s access to Data Cache is blocked if any of the following is true:
|
|||
|
|
|||
|
- MSHR block is full. (The size of MSHR’s buffer is configurable.)
|
|||
|
|
|||
|
- Writeback block is full. (The size of the block’s buffer is
|
|||
|
configurable.)
|
|||
|
|
|||
|
- The number of outstanding memory accesses against the same memory cache line
|
|||
|
has reached configurable threshold value – see MSHR and Write Buffer for details.
|
|||
|
|
|||
|
Data Cache in block state will reject any request from slave port (from CPU)
|
|||
|
regardless of whether it would result in cache hit or miss. Note that
|
|||
|
incoming messages on master port (response messages and snoop requests)
|
|||
|
are never rejected.
|
|||
|
|
|||
|
Cache hit on uncachable memory region (unpredicted behaviour according to
|
|||
|
ARM ARM) will invalidate cache line and fetch data from memory.
|
|||
|
|
|||
|
\subsection gem5_MS_TAndDBlock Tags & Data Block
|
|||
|
|
|||
|
Cache lines (referred as blocks in source code) are organised into sets with
|
|||
|
configurable associativity and size. They have the following status flags:
|
|||
|
- <b>Valid.</b> It holds data. Address tag is valid
|
|||
|
- <b>Read.</b> No read request will be accepted without this flag being set.
|
|||
|
For example, cache line is valid and unreadable when it waits for write flag
|
|||
|
to complete write access.
|
|||
|
- <b>Write.</b> It may accept writes. Cache line with Write flags
|
|||
|
identifies Unique state – no other cache memory holds the copy.
|
|||
|
- <b>Dirty.</b> It needs Writeback when evicted.
|
|||
|
|
|||
|
Read access will hit cache line if address tags match and Valid and Read
|
|||
|
flags are set. Write access will hit cache line if address tags match and
|
|||
|
Valid, Read and Write flags are set.
|
|||
|
|
|||
|
\subsection gem5_MS_Queues MSHR and Write Buffer Queues
|
|||
|
|
|||
|
Miss Status and Handling Register (MSHR) queue holds the list of CPU’s
|
|||
|
outstanding memory requests that require read access to lower memory
|
|||
|
level. They are:
|
|||
|
- Cached Read misses.
|
|||
|
- Cached Write misses.
|
|||
|
- Uncached reads.
|
|||
|
|
|||
|
WriteBuffer queue holds the following memory requests:
|
|||
|
- Uncached writes.
|
|||
|
- Writeback from evicted (& dirty) cache lines.
|
|||
|
|
|||
|
\image html "gem5_MS_Fig3.PNG" "MSHR and Write Buffer Blocks" width=6cm
|
|||
|
|
|||
|
Each memory request is assigned to corresponding MSHR object (READ or WRITE
|
|||
|
on diagram above) that represents particular block (cache line) of memory
|
|||
|
that has to be read or written in order to complete the command(s). As shown
|
|||
|
on gigure above, cached read/writes against the same cache line have a common
|
|||
|
MSHR object and will be completed with a single memory access.
|
|||
|
|
|||
|
The size of the block (and therefore the size of read/write access to lower
|
|||
|
memory) is:
|
|||
|
- The size of cache line for cached access & writeback;
|
|||
|
- As specified in CPU instruction for uncached access.
|
|||
|
|
|||
|
In general, Data Cache model distinguishes between just two memory types:
|
|||
|
- Normal Cached memory. It is always treated as write back, read and write
|
|||
|
allocate.
|
|||
|
- Normal uncached, Device and Strongly Ordered types are treated equally
|
|||
|
(as uncached memory)
|
|||
|
|
|||
|
\subsection gem5_MS_Ordering Memory Access Ordering
|
|||
|
|
|||
|
An unique order number is assigned to each CPU read/write request(as they appear on
|
|||
|
slave port). Order numbers of MSHR objects are copied from the first
|
|||
|
assigned read/write.
|
|||
|
|
|||
|
Memory read/writes from each of these two queues are executed in order (according
|
|||
|
to the assigned order number). When both queues are not empty the model will
|
|||
|
execute memory read from MSHR block unless WriteBuffer is full. It will,
|
|||
|
however, always preserve the order of read/writes on the same
|
|||
|
(or overlapping) memory cache line (block).
|
|||
|
|
|||
|
In summary:
|
|||
|
- Order of accesses to cached memory is not preserved unless they target
|
|||
|
the same cache line. For example, the accesses #1, #5 & #10 will
|
|||
|
complete simultaneously in the same tick (still in order). The access
|
|||
|
#5 will complete before #3.
|
|||
|
- Order of all uncached memory writes is preserved. Write#6 always
|
|||
|
completes before Write#13.
|
|||
|
- Order to all uncached memory reads is preserved. Read#2 always completes
|
|||
|
before Read#8.
|
|||
|
- The order of a read and a write uncached access is not necessarily
|
|||
|
preserved - unless their access regions overlap. Therefore, Write#6
|
|||
|
always completes before Read#8 (they target the same memory block).
|
|||
|
However, Write#13 may complete before Read#8.
|
|||
|
|
|||
|
|
|||
|
\section gem5_MS_Bus COHERENT BUS OBJECT
|
|||
|
|
|||
|
\image html "gem5_MS_Fig4.PNG" "Coherent Bus Object" width=3cm
|
|||
|
|
|||
|
Coherent Bus object provides basic support for snoop protocol:
|
|||
|
|
|||
|
<b>All requests on the slave port</b> are forwarded to the appropriate master port. Requests
|
|||
|
for cached memory regions are also forwarded to other slave ports (as snoop
|
|||
|
requests).
|
|||
|
|
|||
|
<b>Master port replies</b> are forwarded to the appropriate slave port.
|
|||
|
|
|||
|
<b>Master port snoop requests</b> are forwarded to all slave ports.
|
|||
|
|
|||
|
<b>Slave port snoop replies</b> are forwarded to the port that was the source of the
|
|||
|
request. (Note that the source of snoop request can be either slave or
|
|||
|
master port.)
|
|||
|
|
|||
|
The bus declares itself blocked for a configurable period of time after
|
|||
|
any of the following events:
|
|||
|
- A packet is sent (or failed to be sent) to a slave port.
|
|||
|
- A reply message is sent to a master port.
|
|||
|
- Snoop response from one slave port is sent to another slave port.
|
|||
|
|
|||
|
The bus in blocked state rejects the following incoming messages:
|
|||
|
- Slave port requests.
|
|||
|
- Master port replies.
|
|||
|
- Master port snoop requests.
|
|||
|
|
|||
|
\section gem5_MS_SimpleMemory SIMPLE MEMORY OBJECT
|
|||
|
|
|||
|
It never blocks the access on slave port.
|
|||
|
|
|||
|
Memory read/write takes immediate effect. (Read or write is performed when
|
|||
|
the request is received).
|
|||
|
|
|||
|
Reply message is sent after a configurable period of time .
|
|||
|
|
|||
|
\section gem5_MS_MessageFlow MESSAGE FLOW
|
|||
|
|
|||
|
\subsection gem5_MS_Ordering Read Access
|
|||
|
|
|||
|
The following diagram shows read access that hits Data Cache line with Valid
|
|||
|
and Read flags:
|
|||
|
|
|||
|
\image html "gem5_MS_Fig5.PNG" "Read Hit (Read flag must be set in cache line)" width=3cm
|
|||
|
|
|||
|
Cache miss read access will generate the following sequence of messages:
|
|||
|
|
|||
|
\image html "gem5_MS_Fig6.PNG" "Read Miss with snoop reply" width=3cm
|
|||
|
|
|||
|
Note that bus object never gets response from both DCache2 and Memory object.
|
|||
|
It sends the very same ReadReq package (message) object to memory and data
|
|||
|
cache. When Data Cache wants to reply on snoop request it marks the message
|
|||
|
with MEM_INHIBIT flag that tells Memory object not to process the message.
|
|||
|
|
|||
|
\subsection gem5_MS_Ordering Write Access
|
|||
|
|
|||
|
The following diagram shows write access that hits DCache1 cache line with
|
|||
|
Valid & Write flags:
|
|||
|
|
|||
|
\image html "gem5_MS_Fig7.PNG" "Write Hit (with Write flag set in cache line)" width=3cm
|
|||
|
|
|||
|
Next figure shows write access that hits DCache1 cache line with Valid but no
|
|||
|
Write flags – which qualifies as write miss. DCache1 issues UpgradeReq to
|
|||
|
obtain write permission. DCache2::snoopTiming will invalidate cache line that
|
|||
|
has been hit. Note that UpgradeResp message doesn’t carry data.
|
|||
|
|
|||
|
\image html "gem5_MS_Fig8.PNG" "Write Miss – matching tag with no Write flag" width=3cm
|
|||
|
|
|||
|
The next diagram shows write miss in DCache. ReadExReq invalidates cache line
|
|||
|
in DCache2. ReadExResp carries the content of memory cache line.
|
|||
|
|
|||
|
\image html "gem5_MS_Fig9.PNG" "Miss - no matching tag" width=3cm
|
|||
|
|
|||
|
*/
|