Final Project

Overview

Congratulations for making it to the end of the course! We hope this final project gives you a chance to apply what you’ve learned throughout the semester to reason about an application in computing that you’re excited about and/or want to learn more about. In doing so, you will need to struggle with the process of “just getting it working”, develop and test hypotheses about the system, perform some analysis on varying architectures, etc.

We envision two different directions in which you might want to take this final project: gem5-based or or CCV-based. The resources page has some links (including a CCV gearup session recording) to get you started. If you wish to use another tool for your final project, please run it by the course staff to ensure with a plan of action so that we can ensure that it seems feasible in the timeframe remaining in the semester.

Initial abstract and hypotheses (due 4/6, 11pm)

Once you decide on the emerging application you want to explore, write a short abstract and enumerate some hypotheses you want to evaluate. We’re not looking for you to have anticipated all possible nuances of your emerging application/hypotheses, but the more initial effort you put forth here, the better and more detailed our feedback to you will be. The feedback will also be an opportunity to help you calibrate what a “sufficiently complex” project is. We’re looking for hypotheses/emerging applications that are complex enough to demonstrate understanding of the nuances/themes of the course while still controlling for enough variables to make for “good science.”

As you’re planning ahead for your project, keep in mind that a large part of the work will involve setting up experiments in gem5/on Oscar. We don’t expect you to implement the code for your emerging application from scratch – it’s fine to use a well-documented repository here. Note that, in the past, some groups have explored tweaking this code or providing compiler hints as part of a hypothesis that tests the effects of developer intervention on performance.

Abstract

Please write a 5-7 sentence paragraph describing the project. In particular, this paragraph should describe:

  1. What your project is about (~1 sentence)
  2. At a high level, what software and hardware (real or simulated) you want to use for your project (2-3 sentences). In particular, it should be clear from this part of your abstract why the tools that you’re using are relevant to your proposed project.
  3. At a high level, what you are hoping to learn in your evaluation (~1 sentence)

Hypotheses

In more detail, enumerate several (3-5) hypotheses that your evaluation would ideally uncover with respect to the themes that we’ve learned about in the course. A well-formed hypothesis should make an educated claim about an expected, measurable behavior and identify a component of the architecture that would influence that behavior. For example, “I believe that X should outperform Y in terms of throughput for application Z because Z does a which works particularly well on X and not so well on Y because <theme/concept from the course!>

Things you could test for include (but are certainly not limited to!):

  • Whether the performance of your application changes when manipulating specific aspects of the memory hierarchy (cache size, types/number of caches, associativity, virtual memory setup).
  • Whether your application would benefit from dynamic approaches for ILP (deep pipelining, OOO, speculative execution, superscalar, hardware multithreading, etc). If so, which ones, and why? If not, why not? How can you test this?
  • Whether your application would benefit from static approaches for ILP (loop unrolling, VLIW, hand-written optimizations. etc). Note that you should be confident that you can check that your application does/does not take advantage of these.
  • Whether your application would benefit from hardware designed to take advantage of DLP (SIMD instructions, GPUs).
  • Whether the above changes would apply to any configuration/workload of your application, or if the application would benefit from a heuristic approach that chooses between hardware setups.
  • Whether the choice of ISA to compile your application to affects performance, usability, extensibility, etc. of the application
  • Whether an increase in performance of the application is worth an increase in energy/power, cost, compute time, etc. For applications such as machine learning, the answer might be different for the configuration used to train the model vs. the configuration used to run the model.

At least one of your hypotheses should touch on DLP, and at least one should touch on an impact other than performance. There are many ways to think about the impact of our computing setups (resource usage, effect on the developer, effect on the user, security, waste/reuse as new hardware becomes available, etc). Feel free to come by office hours to discuss ideas!

Part of effective experiment design is choosing what aspects of the setup to change (independent variables) and what aspects to keep constant (control variable). A well-written hypothesis will align with an experimental design that has enough control variables to truly test the effect of the independent variable. If you write a hypothesis that tests the effects of dynamic ILP and static ILP and DLP, that might be a sign to split your hypotheses up into multiple pieces. Also consider the complications of testing your hypotheses: if you want to compare how your application performs when compiled to different ISAs, how can you be confident that the specific compiler is actually capable of making the optimizations that your application needs? Again, please stop by office hours if you want to talk more about hypothesis design.

A word of warning on having LLMs help you: In the past, we’ve had groups straight up copy and paste LLM output for the hypotheses, which led to compelling, buzzword-rich statements like “This advantage stems from ARM’s more compact instruction encoding and richer addressing modes. However, x86’s typically higher clock rates and instruction-level optimizations will partially compensate for this instruction count disadvantage” (which, when put under any amount of scrutiny, end up nonsensical and/or false). This is embarrasing (for you). Make sure the hypotheses you turn in actually make sense. Don’t be the group we end up quoting in this warning next year!