1# Bugpoint Redesign 2Author: Diego Treviño (diegotf@google.com) 3 4Date: 2019-06-05 5 6Status: Draft 7 8 9## Introduction 10As use of bugpoint has grown several areas of improvement have been identified 11through years of use: confusing to use, slow, it doesn’t always produce high 12quality test cases, etc. This document proposes a new approach with a narrower 13focus: minimization of IR test cases. 14 15 16## Proposed New Design 17 18 19### Narrow focus: test-case reduction 20The main focus will be a code reduction strategy to obtain much smaller test 21cases that still have the same property as the original one. This will be done 22via classic delta debugging and by adding some IR-specific reductions (e.g. 23replacing globals, removing unused instructions, etc), similar to what 24already exists, but with more in-depth minimization. 25 26 27Granted, if the community differs on this proposal, the legacy code could still 28be present in the tool, but with the caveat of still being documented and 29designed towards delta reduction. 30 31 32### Command-Line Options 33We are proposing to reduce the plethora of bugpoint’s options to just two: an 34interesting-ness test and the arguments for said test, similar to other delta 35reduction tools such as CReduce, Delta, and Lithium; the tool should feel less 36 cluttered, and there should also be no uncertainty about how to operate it. 37 38 39The interesting-ness test that’s going to be run to reduce the code is given 40by name: 41 `--test=<test_name>` 42If a `--test` option is not given, the program exits; this option is similar 43to bugpoint’s current `-compile-custom` option, which lets the user run a 44custom script. 45 46 47The interesting-ness test would be defined as a script that returns 0 when the 48IR achieves a user-defined behaviour (e.g. failure to compile on clang) and a 49nonzero value when otherwise. Leaving the user the freedom to determine what is 50and isn’t interesting to the tool, and thus, streamlining the process of 51reducing a test-case. 52 53 54If the test accepts any arguments (excluding the input ll/bc file), they are 55given via the following flag: 56 `--test_args=<test_arguments>` 57If unspecified, the test is run as given. It’s worth noting that the input file 58would be passed as a parameter to the test, similar how `-compile-custom` 59currently operates. 60 61 62### Implementation 63The tool would behave similar to CReduce’s functionality in that it would have a 64list of passes that try to minimize the given test-case. We should be able to 65modularize the tool’s behavior, as well as making it easier to maintain and 66expand. 67 68 69The first version of this redesign would try to: 70 71 72* Discard functions, instructions and metadata that don’t influence the 73 interesting-ness test 74* Remove unused parameters from functions 75* Eliminate unvisited conditional paths 76* Rename variables to more regular ones (such as “a”, “b”, “c”, etc.) 77 78 79Once these passes are implemented, more meaningful reductions (such as type 80reduction) would be added to the tool, to even further reduce IR. 81 82 83## Background on historical bugpoint issues 84 85 86### Root Cause Analysis 87Presently, bugpoint takes a long time to find the source problem in a given IR 88file, mainly due to the fact that it tries to debug the input by running 89various strategies to classify the bug, which in turn run multiple optimizer 90and compilation passes over the input, taking up a lot of time. Furthermore, 91when the IR crashes, it tries to reduce it by performing some sub-optimal 92passes (e.g. a lot of unreachable blocks), and sometimes even fails to minimize 93at all. 94 95 96### "Quirky" Interface 97Bugpoint’s current interface overwhelms and confuses the user, the help screen 98alone ends up confusing rather providing guidance. And, not only are there 99numerous features and options, but some of them also work in unexpected ways 100and most of the time the user ends up using a custom script. Pruning and 101simplifying the interface will be worth considering in order to make the tool 102more useful in the general case and easier to maintain. 103