Final Project

Part 0: Overview

Congratulations on making it to the end of the course! We hope this final project gives you a chance to apply what you’ve learned throughout the semester to reason about an application in computing that you’re excited about and/or want to learn more about. You will work in groups to analyze the performance of emerging computer science applications. The applications should be recent, fairly computationally complex, and have code available. Along the way, you will need to struggle the process of “just getting it working”, developing and testing hypotheses about the system, performing some analysis on varying architectures, etc.

We envision two different directions in which you might want to take this final project: gem5-based, in which we will refer you to the gem5 resources for direction to get started; or CCV-based, in which you may find it useful to attend/refer to the final project gear-up. If you wish to use another tool for your final project, please run it by the course staff to ensure with a plan of action so that we can ensure that it seems feasible in the timeframe remaining in the semester.

Initial submission

By 10pm on Friday, April 19, please submit a drat of your abstract and hypotheses. The hypotheses can be in bulleted list format and don’t have to be completely polished. We just want the opportunity to give you early feedback on the direction of your work before you get started instrumenting your experiments!

Final Submission

By 10pm on Monday, May 6, please upload a PDF that describes your answers through the project steps, and any relevant results and/or instrumenting that you put together to Gradescope.

Part 1: Project Abstract

Please write a 5-7 sentence paragraph describing the project. In particular, this paragraph should describe:

  1. What it is that your project is about (~1 sentence)
  2. At a high level, what software and hardware (real or simulated) do you want to use for your project (2-3 sentences). In particular, it should be clear from this part of your abstract why the tools that you’re using are relevant to your proposed project.
  3. At a high level, what are you hoping to learn in your evaluation?

Part 2: Hypotheses

In more detail, enumerate several (3-5) hypotheses that your evaluation would ideally uncover with respect to the themes that we’ve learned about in the course. Note, a good hypothesis should make an educated claim about an expected, measurable behavior and identify a component of the architecture that would influence that behavior. For example, “I believe that X should outperform Y in terms of throughput for application Z because Z does a which works particularly well on X and not so well on Y because <theme/concept from the course!>

We will give you some initial feedback on your hypotheses early on in the project. Things you could test for include (but are certainly not limited to!):

  • Whether the performance of your application changes when manipulating specific aspects of the memory hierarchy (cache size, types/number of caches, associativity, virtual memory setup).
  • Whether your application would benefit from dynamic approaches for ILP (deep pipelining, OOO, speculative execution, superscalar, hardware multithreading, etc). If so, which ones, and why? If not, why not? How can you test this?
  • Whether your application would benefit from static approaches for ILP (loop unrolling, VLIW, hand-written optimizations. etc). Note that you should be confident that you can check that your application does/does not take advantage of these.
  • Whether your application would benefit from hardware designed to take advantage of DLP (SIMD instructions, GPUs).
  • Whether the above changes would apply to any configuration/workload of your application, or if the application would benefit from a heuristic approach that chooses between hardware setups.
  • Whether the choice of ISA to compile your application to affects performance, usability, extensibility, etc. of the application
  • Whether an increase in performance of the application is worth an increase in energy/power, cost, compute time, etc. For applications such as machine learning, the answer might be different for the configuration used to train the model vs. the configuration used to run the model.

Notice that some of these examples are concerned with the domain of social responsibility. Please write at least one hypothesis that is related to this domain. There are many ways to think about the impact of our computing setups (resource usage, effect on the developer, effect on the user, security, waste/reuse as new hardware becomes available, etc). Feel free to come by office hours to discuss ideas!

Part of effective experiment design is choosing what aspects of the setup to change (independent variables) and what aspects to keep constant (control variable). A well-written hypothesis will align with an experimental design that has enough control variables to truly test the effect of the independent variable. If you write a hypothesis that tests the effects of dynamic ILP and static ILP and DLP, for example, that might be a sign to split your hypotheses up into multiple pieces. Also consider the complications of testing your hypotheses: if you want to compare how your application performs when compiled to different ISAs, how can you be confident that the specific compiler is actually capable of making the optimizations that your application needs? Again, please stop by office hours if you want to talk more about hypothesis design.

Part 3: Putting it all together!

Time to start coding! As you set up your environment, take some notes about the details of your thought process. For example, some questions you may consider explaining: how many CPUs do you want to use? GPUs? Which types of these devices are you using? Do you think making this decision would matter? What does your memory hierarchy look like?

In addition, working with open source projects can be a bit of a pain sometimes, and just getting stuff up and running can be tricky. Please enumerate the steps it takes to get your project up and running here so that we can empathize with any potential struggle you may endure!

Finally, once your environment is together, please describe any scaffolding and/or run scripts that you put together to be able to perform your evaluation. This may include scripts to run stuff on CCV, in gem5, etc. or to directly call components from the open source project.

Part 4: Evaluating your hypotheses

Now that your environment is together, time to test your hypotheses!

First, describe what tests you want to run to test each hypothesis. As mentioned in part 2, a good test is one that controls for as many variables in the architecture as possible, and leaves only the thing that you want to test as variable (this is easier in a simulator than on real hardware).

Afterwards, summarize your evaluation. What result did you find? Did it confirm your hypothesis? Deny it? Leave results inconclusive? Use whatever evidence you need to back up your findings, such as figures, tables, etc. You don’t need pages of writing here – just enough so that a reader (us) can buy the argument you’re making.

As you are working on parts 3-4, if you find yourself running into setup issues or other setbacks, you might find that you need to re-scope the work or change your approach in some way. That’s okay! If you’re super stuck, please reach out to us (the sooner you do, the better we can help you and the less stress there will be). Regardless, please document and explain any pivots you had to make in order to perform your analysis. It’s fine to scale back the project, but your explanations should demonstrate understanding of what was difficult about the original setup and why you decided to go in the new direction.

Part 5: Summary

You worked hard on the project, so use this section to brag about what you learned (in about a paragraph! We want to see some big-picture reflections here, i.e.:

  1. What are the major design tradeoff(s) you explored? Are there tradeoffs you didn’t get to explore?
  2. Do you see future directions in the field of computer architecture aligning with the observations you made in your analysis?
  3. What would you differently with your analysis if you had access to any hardware currently on the market? Are there any wild and wacky architectures out there that would suit the needs of your application?