[P1] Streaming memory unit

[tancheng]

We can enable streaming loading/storing inside the existing memory unit, or creating a standalone unit.

• It should be in 2 cycles for loading in terms of latency, but should be able to pipeline.
• It can just occupy the entire tile without temporal reconfigurability.

[yyan7223]

• It should be in 2 cycles for loading in terms of latency, but should be able to pipeline.

Does streaming loading/storing mean that we can keep sending different waddr&wdata, raddr to memory unit at adjacent clock cycles, without blocking here to wait for the result of a single memory access?
Pipelining may requires input queues for waddr&wdata, raddr, and everytime mem unit finishes a ld/st request, it fetches a new request from the queue.

• It can just occupy the entire tile without temporal reconfigurability.

Is this for the future plan of spatial-only CGRA?

[tancheng]

Pipelining may requires input queues for waddr&wdata, raddr, and everytime mem unit finishes a ld/st request, it fetches a new request from the queue.

Sounds good to me. Yes, a queue or fifo with two entries is good enough. Enqueue address and address + 1 till address + bound in each cycle.

Is this for the future plan of spatial-only CGRA?

It could also be spatial-temporal CGRA but leverage one specific tile in spatial-only way (i.e., only one op in that tile).

[yyan7223]

Hi @tancheng ,

Seems that code below in DataMemControllerRTL has already implemented streaming?

VectorCGRA/mem/data/DataMemControllerRTL.py

Lines 130 to 135 in 8b7da62

	s.read_crossbar = XbarBypassQueueRTL(MemReadPktType, num_xbar_in_rd_ports,
	num_xbar_out_rd_ports)
	s.write_crossbar = XbarBypassQueueRTL(MemWritePktType, num_xbar_in_wr_ports,
	num_xbar_out_wr_ports)
	s.response_crossbar = XbarBypassQueueRTL(MemResponsePktType, num_xbar_out_rd_ports,
	num_xbar_in_rd_ports)

[tancheng]

Hi @tancheng ,

Seems that code below in DataMemControllerRTL has already implemented streaming?

VectorCGRA/mem/data/DataMemControllerRTL.py

Lines 130 to 135 in 8b7da62

s.read_crossbar = XbarBypassQueueRTL(MemReadPktType, num_xbar_in_rd_ports,
num_xbar_out_rd_ports)
s.write_crossbar = XbarBypassQueueRTL(MemWritePktType, num_xbar_in_wr_ports,
num_xbar_out_wr_ports)
s.response_crossbar = XbarBypassQueueRTL(MemResponsePktType, num_xbar_out_rd_ports,
num_xbar_in_rd_ports)

This is orthogonal to streaming, though it can be used. This is just enqueuing the requests one by one. I originally thought streaming request might be a single request, instead of multiple ones, but the responses could be multiple. Or you can have a streaming_ld unit keep issuing ld requests.

The second solution is easier in terms of implementation, but I am not sure what is the typical way. Can you do a rough survey/investigation on this?

[tancheng]

I personally prefer to going for the second solution after talking with GPT :-) The first solution requires bank holding, arbitration, etc.

[yyan7223]

Oh I revisit our wechat conversation.
you mean raddr/waddr should contain new fields like start, stride, end, then automatically streaming data.
So this may require extending exisiting packet with start, stride, end

VectorCGRA/lib/messages.py

Line 373 in 8b7da62

def mk_mem_access_pkt(DataType,

and an module here to automatically generate raddr according to start, stride, end.

VectorCGRA/mem/data/DataMemWrapperRTL.py

Line 80 in 8b7da62

s.memory.raddr[0] @= \

Streaming sounds reasonable for reading, one reading request can return multiple responses.
How about writing? Even if waddr contains start, stride, end, we might still need to contain multiple wdata in multiple packets.

[tancheng]

Let's just have a streaming read then? I am not able to think about streaming write from CGRA fine-granularity use case.

[yyan7223]

Yeah sure