Chan z prefix sum

[amin1377]

Description

This PR optimizes the calculation of the ChanZ cost factor by precomputing and storing the cumulative number of inter-die connections prior to block placement. During placement, the number of inter-die connections within a bounding box is calculated using the following formula:

if (x_low == 0 && y_low == 0) {
    num_inter_dir_conn = acc_tile_num_inter_die_conn_[x_high][y_high];
} else if (x_low == 0) {
    num_inter_dir_conn = acc_tile_num_inter_die_conn_[x_high][y_high] - 
                         acc_tile_num_inter_die_conn_[x_high][y_low - 1];
} else if (y_low == 0) {
    num_inter_dir_conn = acc_tile_num_inter_die_conn_[x_high][y_high] - 
                         acc_tile_num_inter_die_conn_[x_low - 1][y_high];
} else {
    num_inter_dir_conn = acc_tile_num_inter_die_conn_[x_high][y_high] - 
                         acc_tile_num_inter_die_conn_[x_low - 1][y_high] - 
                         acc_tile_num_inter_die_conn_[x_high][y_low - 1] + 
                         acc_tile_num_inter_die_conn_[x_low - 1][y_low - 1];
}

[vaughnbetz]

1. You should comment how the prefix sum computation works.
2. We can't afford an O(n^2) memory and O(n^2) compute to load this data structure, as N is the size of the circuit. We shouldn't compute the final matrix, but instead just store the O(n) (2D in x,y) prefix sum. Compute the average z-capacity in a helper function when called.

[vaughnbetz]

QoR results are at https://www.eecg.utoronto.ca/~vaughn/vaughnwiki/doku.php?id=amin_weekly_update and look good. On both 3D SW and 2D Titan architectures, pack, place and route time and QoR are essentially unchanged, and on a 3D arch total runtime drops by 2%, which makes sense as the loading of the cost structures should be faster (and may be counted outside place time). @amin1377 : if you can paste the QoR data in for posterity that would be good.

[vaughnbetz]

Some commenting updates, and one functional change to divide by layer-1.

[soheilshahrouz]

does tile_num_inter_die_conn need to store float values?

[vaughnbetz]

It's a count, so int is probably better. Float could have some round-off issues with big chips, as this matrix is filled in by adding small numbers to a running count, which can get problematic if the big number ever becomes more than 16 million times or so bigger than the small number (the small number then gets thrown away by round off).

[soheilshahrouz]

using vtr::NdOffsetMatrix will be consistent with thest of file and will get rid of this if statement.

[vaughnbetz]

Yes, that would be cleaner.

[amin1377]

@soheilshahrouz, if you remember, my initial attempt was to use this data structure, but I encountered some memory issues with it. I’ll create an issue for this and, in another PR, attempt to replace this data structure with vtr::NdOffsetMatrix.

[soheilshahrouz]

I think I figured out what the problem was.
Check if 68ecb55 sovles the memory issue.

[amin1377]

Sounds good. I'll try it once the PR is merged.

[vaughnbetz]

This is separate from this PR, but it's possible that using the same technique in the x- and y- directions that you're using for the z-direction to calculate the average channel width by adding and subtracting the proper values would be faster for x- and y- too.

It saves memory (would take storage for x and y average channel capacities down from O(device size) to O(sqrt(device size)), which would help the cache and memory bandwidth. If would mean two matrix accesses instead of 1, and an extra reciprocal and pow() call, but it might still be overall just as fast or faster. After this PR is landed, if you have th energy for it you could do a quick experiment to check by recoding the x- and y- dimensions to match the z-dimension approach.

[soheilshahrouz]

This is separate from this PR, but it's possible that using the same technique in the x- and y- directions that you're using for the z-direction to calculate the average channel width by adding and subtracting the proper values would be faster for x- and y- too.

@amin1377 If you're busy with other tasks, I am interested in working on this idea for chan_x and chan_y. Let me know if I can take it over.

[amin1377]

This is separate from this PR, but it's possible that using the same technique in the x- and y- directions that you're using for the z-direction to calculate the average channel width by adding and subtracting the proper values would be faster for x- and y- too.

@amin1377 If you're busy with other tasks, I am interested in working on this idea for chan_x and chan_y. Let me know if I can take it over.

How could I possibly say no to such a generous offer, Soheil? Thanks a lot! :) Let me know if I can be of any help.

[amin1377]

QoR:
titan other 3d CB cube bb

titan_quick_qor

[vaughnbetz]

QoR looks good.

[amin1377]

@vaughnbetz, I’ve addressed both your comments and Soheil’s. After making the changes, I reran the benchmarks to ensure everything is still working correctly, and the tests passed without issues. I think the PR is ready to be merged.