1*7330f729Sjoerg# Bugpoint Redesign 2*7330f729SjoergAuthor: Diego Treviño (diegotf@google.com) 3*7330f729Sjoerg 4*7330f729SjoergDate: 2019-06-05 5*7330f729Sjoerg 6*7330f729SjoergStatus: Draft 7*7330f729Sjoerg 8*7330f729Sjoerg 9*7330f729Sjoerg## Introduction 10*7330f729SjoergAs use of bugpoint has grown several areas of improvement have been identified 11*7330f729Sjoergthrough years of use: confusing to use, slow, it doesn’t always produce high 12*7330f729Sjoergquality test cases, etc. This document proposes a new approach with a narrower 13*7330f729Sjoergfocus: minimization of IR test cases. 14*7330f729Sjoerg 15*7330f729Sjoerg 16*7330f729Sjoerg## Proposed New Design 17*7330f729Sjoerg 18*7330f729Sjoerg 19*7330f729Sjoerg### Narrow focus: test-case reduction 20*7330f729SjoergThe main focus will be a code reduction strategy to obtain much smaller test 21*7330f729Sjoergcases that still have the same property as the original one. This will be done 22*7330f729Sjoergvia classic delta debugging and by adding some IR-specific reductions (e.g. 23*7330f729Sjoergreplacing globals, removing unused instructions, etc), similar to what 24*7330f729Sjoergalready exists, but with more in-depth minimization. 25*7330f729Sjoerg 26*7330f729Sjoerg 27*7330f729SjoergGranted, if the community differs on this proposal, the legacy code could still 28*7330f729Sjoergbe present in the tool, but with the caveat of still being documented and 29*7330f729Sjoergdesigned towards delta reduction. 30*7330f729Sjoerg 31*7330f729Sjoerg 32*7330f729Sjoerg### Command-Line Options 33*7330f729SjoergWe are proposing to reduce the plethora of bugpoint’s options to just two: an 34*7330f729Sjoerginteresting-ness test and the arguments for said test, similar to other delta 35*7330f729Sjoergreduction tools such as CReduce, Delta, and Lithium; the tool should feel less 36*7330f729Sjoerg cluttered, and there should also be no uncertainty about how to operate it. 37*7330f729Sjoerg 38*7330f729Sjoerg 39*7330f729SjoergThe interesting-ness test that’s going to be run to reduce the code is given 40*7330f729Sjoergby name: 41*7330f729Sjoerg `--test=<test_name>` 42*7330f729SjoergIf a `--test` option is not given, the program exits; this option is similar 43*7330f729Sjoergto bugpoint’s current `-compile-custom` option, which lets the user run a 44*7330f729Sjoergcustom script. 45*7330f729Sjoerg 46*7330f729Sjoerg 47*7330f729SjoergThe interesting-ness test would be defined as a script that returns 0 when the 48*7330f729SjoergIR achieves a user-defined behaviour (e.g. failure to compile on clang) and a 49*7330f729Sjoergnonzero value when otherwise. Leaving the user the freedom to determine what is 50*7330f729Sjoergand isn’t interesting to the tool, and thus, streamlining the process of 51*7330f729Sjoergreducing a test-case. 52*7330f729Sjoerg 53*7330f729Sjoerg 54*7330f729SjoergIf the test accepts any arguments (excluding the input ll/bc file), they are 55*7330f729Sjoerggiven via the following flag: 56*7330f729Sjoerg `--test_args=<test_arguments>` 57*7330f729SjoergIf unspecified, the test is run as given. It’s worth noting that the input file 58*7330f729Sjoergwould be passed as a parameter to the test, similar how `-compile-custom` 59*7330f729Sjoergcurrently operates. 60*7330f729Sjoerg 61*7330f729Sjoerg 62*7330f729Sjoerg### Implementation 63*7330f729SjoergThe tool would behave similar to CReduce’s functionality in that it would have a 64*7330f729Sjoerglist of passes that try to minimize the given test-case. We should be able to 65*7330f729Sjoergmodularize the tool’s behavior, as well as making it easier to maintain and 66*7330f729Sjoergexpand. 67*7330f729Sjoerg 68*7330f729Sjoerg 69*7330f729SjoergThe first version of this redesign would try to: 70*7330f729Sjoerg 71*7330f729Sjoerg 72*7330f729Sjoerg* Discard functions, instructions and metadata that don’t influence the 73*7330f729Sjoerg interesting-ness test 74*7330f729Sjoerg* Remove unused parameters from functions 75*7330f729Sjoerg* Eliminate unvisited conditional paths 76*7330f729Sjoerg* Rename variables to more regular ones (such as “a”, “b”, “c”, etc.) 77*7330f729Sjoerg 78*7330f729Sjoerg 79*7330f729SjoergOnce these passes are implemented, more meaningful reductions (such as type 80*7330f729Sjoergreduction) would be added to the tool, to even further reduce IR. 81*7330f729Sjoerg 82*7330f729Sjoerg 83*7330f729Sjoerg## Background on historical bugpoint issues 84*7330f729Sjoerg 85*7330f729Sjoerg 86*7330f729Sjoerg### Root Cause Analysis 87*7330f729SjoergPresently, bugpoint takes a long time to find the source problem in a given IR 88*7330f729Sjoergfile, mainly due to the fact that it tries to debug the input by running 89*7330f729Sjoergvarious strategies to classify the bug, which in turn run multiple optimizer 90*7330f729Sjoergand compilation passes over the input, taking up a lot of time. Furthermore, 91*7330f729Sjoergwhen the IR crashes, it tries to reduce it by performing some sub-optimal 92*7330f729Sjoergpasses (e.g. a lot of unreachable blocks), and sometimes even fails to minimize 93*7330f729Sjoergat all. 94*7330f729Sjoerg 95*7330f729Sjoerg 96*7330f729Sjoerg### "Quirky" Interface 97*7330f729SjoergBugpoint’s current interface overwhelms and confuses the user, the help screen 98*7330f729Sjoergalone ends up confusing rather providing guidance. And, not only are there 99*7330f729Sjoergnumerous features and options, but some of them also work in unexpected ways 100*7330f729Sjoergand most of the time the user ends up using a custom script. Pruning and 101*7330f729Sjoergsimplifying the interface will be worth considering in order to make the tool 102*7330f729Sjoergmore useful in the general case and easier to maintain. 103