Skip to content

Commit 78c94bc

Browse files
tgross35tgross35
committed
fuzz: Improve flexibility of the fuzz binary
There are a number of chnages here that aren't easy to split up: * Accept rounding mode as an input, and check status flags as an output. * Catch C++ exceptions (if enabled) in C++ rather than relying on `catch_unwind` in Rust. * A `DecodeError` type is introduced for propagating parse failures, i.e. when the fuzzer gives us an invalid binary. This means `catch_unwind` can be eliminated and panics can be reserved for failures we want the fuzzer to detect. * Rename "hard" to "host" since host floats can be softfloat. * Ignore NaN mismatches with reasons, rather than fixing up the outputs. * Similarly ignore NaN mismatches when the input is signaling and there is a F1->F2->F1 roundtrip with F1 and F2 as the same float, since LLVM may do nothing here. * Add a CLI option to check a single file like the fuzzer does. This introduces some code duplication since the brute force tests are mostly untouched, and those use some of the same logic. A future commit will clean this up. There are some failures here that may be resolved in future LLVM commits. Examples: fuzz/out-foo5/default/crashes/id:000027,sig:06,src:000855,time:629681,execs:17464051,op:havoc,rep:1: f16.MulAdd(0x23ff /* 0.015617 */, 0x17ff /* 0.0019522 */, 0x80ff /* -1.5199E-5 */, NearestTiesToEven) => 0x0101 /* 1.5318E-5 */ Status(UNDERFLOW | INEXACT) (Rust / rustc_apfloat) => 0x0100 /* 1.5259E-5 */ Status(UNDERFLOW | INEXACT) (C++ / llvm::APFloat) <- !!! MISMATCH !!! LLVM has the incorrect result here. fuzz/sync/fuzzer02/crashes/id:000021,sig:06,src:001760,time:533420,execs:9318705,op:havoc,rep:1: brainf16.MulAdd(0xbe01 /* -0.126 */, 0x007f /* 1.166E-38 */, 0x0010 /* 1.469E-39 */, NearestTiesToAway) => 0x0000 /* 0 */ Status(UNDERFLOW | INEXACT) (Rust / rustc_apfloat) => 0x0001 /* 9.184E-41 */ Status(UNDERFLOW | INEXACT) (C++ / llvm::APFloat) <- !!! MISMATCH !!! Similarly, LLVM is not correct. fuzz/out-06/default/crashes/id:000000,sig:06,src:000060,time:1970,execs:39760,op:havoc,rep:2: f32.FToSingleToF(0xff800080 /* NaN */, NearestTiesToEven) => 0xffc00080 /* NaN */ Status(0x0) (Rust / rustc_apfloat) => 0xff800080 /* NaN */ Status(0x0) (C++ / llvm::APFloat) <- !!! MISMATCH !!! => 0xff800080 /* NaN */ Status(0x0) (native host floats) <- !!! MISMATCH (ignored, both are propagated inputs) !!! Here Rust quiets the input sNaN. This seems like correct behavior since usually conversions on NaN do make it quiet, but it seems like for LLVM and the host floats it turns into a no-op since the size is the same. fuzz/sync/fuzzer02/crashes/id:000016,sig:06,src:000190,time:255651,execs:2135435,op:havoc,rep:2: f8e4m3fn.Add(0xfd /* -416 */, 0xe8 /* -64 */, TowardNegative) => 0xfe /* -448 */ Status(INEXACT) (Rust / rustc_apfloat) => 0xff /* NaN */ Status(OVERFLOW | INEXACT) (C++ / llvm::APFloat) <- !!! MISMATCH (ignored, f8e4m3fn may be broken) !!! Various cases for f8e4m3fn fail, including the below that crashes in LLVM APFloat: target/fuzz-out/fuzzer07/crashes/id:000000,sig:06,src:000629,time:584123,execs:2204007,op:havoc,rep:1: f8e4m3fn.MulAdd(0x9d /* -0.102 */, 0x0a /* 0.0195 */, 0x01 /* 0.00195 */, NearestTiesToEven) => 0x80 /* -0 */ Status(UNDERFLOW | INEXACT) (Rust / rustc_apfloat) In a future commit, these can all be added to the corpus.
1 parent debb27c commit 78c94bc

7 files changed

Lines changed: 1299 additions & 143 deletions

File tree

‎README.md‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ allocating to handle the arbitrary precision needed for conversions to/from deci
1818

1919
However, that port had a fatal flaw: it was added to the `rust-lang/rust` repository
2020
without its unique licensing status (as a port of a C++ library with its own license)
21-
being properly tracked, communicated, taken into account, etc.
21+
being properly tracked, communicated, taken into account, etc.
2222
The end result was years of limbo, mostly chronicled in the Rust issue
2323
[`rust-lang/rust#55993`](https://github.com/rust-lang/rust/issues/55993), in which
2424
the in-tree port couldn't really receive proper updated or even maintenance, due
@@ -28,39 +28,39 @@ due to its unclear status.
2828

2929
This repository (`rust-lang/rustc_apfloat`) is the result of a 2022 plan on
3030
[the relevant Zulip topic](https://rust-lang.zulipchat.com/#narrow/stream/231349-t-core.2Flicensing/topic/apfloat), fully put into motion during 2023:
31-
* the `git` history of the in-tree `compiler/rustc_apfloat` library was extracted
31+
* the `git` history of the in-tree `compiler/rustc_apfloat` library was extracted
3232
(see the separate [`rustc_apfloat-git-history-extraction`](https://github.com/LykenSol/rustc_apfloat-git-history-extraction) repository for more details)
3333
* only commits that were *both* necessary *and* had clear copyright status, were kept
34-
* any missing functionality or bug fixes, would have to be either be re-contributed,
34+
* any missing functionality or bug fixes, would have to be either be re-contributed,
3535
or rebuilt from the ground up (mostly the latter ended up being done, see below)
3636

3737
Most changes since the original port had been aesthetic (e.g. spell-checking, `rustfmt`),
3838
so little was lost in the process.
3939

4040
Starting from that much smaller "trusted" base:
41-
* everything could use LLVM's new (since 2019) license, "`Apache-2.0 WITH LLVM-exception`"
41+
* everything could use LLVM's new (since 2019) license, "`Apache-2.0 WITH LLVM-exception`"
4242
(see the ["Licensing"](#licensing) section below and/or [LICENSE-DETAILS.md](./LICENSE-DETAILS.md) for more details)
4343
* new facilities were built (benchmarks, and [a fuzzer comparing Rust/C++/hardware](#fuzzing))
4444
* excessive testing was performed (via a combination of fuzzing and bruteforce search)
4545
* latent bugs were discovered (e.g. LLVM issues
4646
[#63895](https://github.com/llvm/llvm-project/issues/63895) and
4747
[#63938](https://github.com/llvm/llvm-project/issues/63938))
48-
* the port has been forwarded in time, to include upstream (`llvm/llvm-project`) changes
48+
* the port has been forwarded in time, to include upstream (`llvm/llvm-project`) changes
4949
to `llvm::APFloat` over the years (since 2017), removing the need for selective backports
5050

5151
## Versioning
5252

53-
As this is, for the time being, a "living port", tracking upstream (`llvm/llvm-project`)
53+
As this is, for the time being, a "living port", tracking upstream (`llvm/llvm-project`)
5454
`llvm::APFloat` changes, the `rustc_apfloat` crate will have versions of the form:
5555

5656
```
5757
0.X.Y+llvm-ZZZZZZZZZZZZ
5858
```
59-
* `X` is always bumped after semver-incompatible API changes,
59+
* `X` is always bumped after semver-incompatible API changes,
6060
or when updating the upstream (`llvm/llvm-project`) commit the port is based on
6161
* `Y` is only bumped when other parts of the version don't need to be (e.g. for bug fixes)
62-
* `+llvm-ZZZZZZZZZZZZ` is ["version metadata"](https://doc.rust-lang.org/cargo/reference/resolver.html#version-metadata) (which Cargo itself ignores),
63-
and `ZZZZZZZZZZZZ` always holds the first 12 hexadecimal digits of
62+
* `+llvm-ZZZZZZZZZZZZ` is ["version metadata"](https://doc.rust-lang.org/cargo/reference/resolver.html#version-metadata) (which Cargo itself ignores),
63+
and `ZZZZZZZZZZZZ` always holds the first 12 hexadecimal digits of
6464
the upstream (`llvm/llvm-project`) `git` commit hash the port is based on
6565

6666

@@ -84,7 +84,7 @@ involves an automated build of the original C++ `llvm::APFloat` code with `clang
8484
Rust code), and has been prototyped and tested on Linux (and is unlikely to work
8585
on other platforms, or even some Linux distros, though it mostly assumes UNIX).
8686

87-
Example usage:
87+
Example usage:
8888
<sub>(**TODO**: maybe move this to `fuzz/README.md` and/or expand on it)</sub>
8989

9090
```sh
@@ -103,10 +103,15 @@ cargo afl fuzz -i fuzz/in-foo -o fuzz/out-foo target/release/rustc_apfloat-fuzz
103103
```
104104

105105
To visualize the fuzzing testcases, you can use the `decode` subcommand:
106+
106107
```sh
107108
cargo run -p rustc_apfloat-fuzz decode fuzz/out-foo/default/crashes/*
108109
```
109-
(this will work even while `cargo afl fuzz`, i.e. AFL++, is running)
110+
111+
Note that `cargo run` and `cargo afl build` conflict, so if running the fuzzer
112+
and then decoding with the same debug/release setting, this will always trigger
113+
a rebuild. In these cases, launching the binary directly to call `decode` can
114+
avoid the extra builds.
110115

111116
## Licensing
112117

‎etc/fuzz-parallel.sh‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
#!/bin/bash
2+
3+
# Launch the fuzzer with multiple parallel jobs. Requires tmux.
4+
#
5+
# Taken from: <https://github.com/rust-fuzz/afl.rs/issues/132#issuecomment-997827086>
6+
7+
set -euxo pipefail
8+
9+
# Detect cores
10+
all_cores="$(nproc)"
11+
used_cores="$((all_cores - 2))"
12+
in_dir="target/fuzz-in"
13+
sync_dir="target/fuzz-out"
14+
tmux_window=afl
15+
16+
if [[ "$used_cores" -lt 2 ]]; then
17+
echo "Error: used_cores < 2"
18+
exit 1
19+
fi
20+
21+
function print_usage() {
22+
set +x
23+
echo "Usage: $0 [-j PROCS]"
24+
echo ""
25+
echo "Options:"
26+
echo " -o DIR Output directory"
27+
echo " -j PROCS Number of parallel jobs to use (default: $used_cores)"
28+
echo " -h,--help Print this help and exit"
29+
set -x
30+
}
31+
32+
# Parse arguments
33+
while [[ "$#" -gt 0 ]]; do
34+
case $1 in
35+
-j) used_cores="$2"; shift ;;
36+
-o) sync_dir="$2"; shift ;;
37+
-h|--help) print_usage; exit 0 ;;
38+
esac
39+
shift
40+
done
41+
42+
echo "Using $used_cores out of $all_cores cores"
43+
44+
45+
# Make sure we have at least one input file
46+
mkdir -p "$in_dir"
47+
echo > "$in_dir/empty"
48+
49+
# Start main node
50+
tmux new -d -s "afl01" -n $tmux_window \
51+
"cargo afl fuzz -i $in_dir -o $sync_dir -M fuzzer01 target/release/rustc_apfloat-fuzz"
52+
echo "Spawned main instance afl01"
53+
54+
# Start secondary instances
55+
for i in $(seq -f "%02.0f" 2 "$used_cores"); do
56+
tmux new -d -s "afl$i" -n $tmux_window \
57+
cargo afl fuzz -i $in_dir -o "$sync_dir" -S "fuzzer$i" target/release/rustc_apfloat-fuzz
58+
echo "Spawned secondary instance afl$i"
59+
done
60+
61+
set +x
62+
63+
# Show status output
64+
echo ""
65+
echo "Tmux sessions:"
66+
tmux ls | grep afl
67+
echo ""
68+
echo "Tmux cheatsheet (shell):"
69+
echo " Attach:"
70+
echo " tmux attach -t afl01"
71+
echo " Kill all sessions:"
72+
echo " tmux kill-server"
73+
echo ""
74+
echo "Tmux chatsheet (inside tmux):"
75+
echo " List sessions:"
76+
echo " Ctrl-b s"
77+
echo " Switch to next session:"
78+
echo " Ctrl-b )"
79+
echo " Switch to prev session:"
80+
echo " Ctrl-b ("
81+
echo " Detach:"
82+
echo " Ctrl-b d"

‎fuzz/Cargo.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,7 @@ rustc_apfloat = { path = ".." }
1414
unexpected_cfgs = { level = "warn", check-cfg = [
1515
# Set by the fuzzer
1616
'cfg(fuzzing)',
17+
# Unstable configs
18+
'cfg(target_has_reliable_f16)',
19+
'cfg(target_has_reliable_f128)',
1720
] }

‎fuzz/build.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ use std::process::Command;
66
// NB: Any new symbols exported from the C++ source file need to be listed here,
77
// everything else will get pruned.
88
const CXX_EXPORTED_SYMBOLS: &[&str] = &[
9+
"check_error",
910
"cxx_apf_eval_op_brainf16",
1011
"cxx_apf_eval_op_ieee16",
1112
"cxx_apf_eval_op_ieee32",

‎fuzz/src/apf_fuzz.cpp‎

Lines changed: 49 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11

22
#include <array>
33
#include <cstdint>
4+
#include <string.h>
45
#include <stdint.h>
56
#include <stdio.h>
67
#include "llvm/ADT/APFloat.h"
@@ -43,11 +44,11 @@ enum class OpCode: uint8_t {
4344

4445
/** Similarly, rounding mode is passed as a u8 */
4546
enum class Round: uint8_t {
46-
NearestTiesToEven = 0,
47-
TowardZero = 1,
48-
TowardPositive = 2,
49-
TowardNegative = 3,
50-
NearestTiesToAway = 4,
47+
NearestTiesToEven = 0,
48+
TowardZero = 1,
49+
TowardPositive = 2,
50+
TowardNegative = 3,
51+
NearestTiesToAway = 4,
5152
};
5253

5354
/* LLVM uses the following values:
@@ -57,8 +58,23 @@ enum class Round: uint8_t {
5758
* opOverflow = 0x04,
5859
* opUnderflow = 0x08,
5960
* opInexact = 0x10
61+
*
62+
* We also use -1 to indicate an exception.
6063
*/
61-
using StatusFlags = unsigned;
64+
using StatusFlags = int32_t;
65+
66+
/* Utilities for making C++ exceptions FFI-safe */
67+
thread_local const char *AP_ERROR = NULL;
68+
69+
StatusFlags handle_std_exception(const std::exception& e) {
70+
AP_ERROR = strdup(e.what());
71+
return -1;
72+
}
73+
74+
StatusFlags handle_unknown_exception() {
75+
AP_ERROR = "Unknown exception";
76+
return -1;
77+
}
6278

6379
/** Common operations for a given semantics are grouped in this class */
6480
template<APFloat::Semantics S, typename U, const unsigned BITS = sizeof(U) * 8>
@@ -96,8 +112,7 @@ class FloatEval {
96112
return APFloat(getSemantics(), APInt(BITS, words));
97113
}
98114

99-
/** Evaluate a dynamically specified operation with the given configuration */
100-
static StatusFlags eval(OpCode op, Round round, UInt ai, UInt bi,
115+
static StatusFlags eval_impl(OpCode op, Round round, UInt ai, UInt bi,
101116
UInt ci, UInt &out)
102117
{
103118
APFloat a = bits_to_apf(ai);
@@ -154,7 +169,7 @@ class FloatEval {
154169
status = a.fusedMultiplyAdd(b, c, rm);
155170
break;
156171
/* FIXME: the below operations could be incorrect and are discarding a
157-
status, and (though unlikely) could have mistkes that cancel. It would
172+
status, and (though unlikely) could have mistakes that cancel. It would
158173
be better to make `out` a u128 and only do a single conversion. */
159174
case OpCode::FToI128ToF:
160175
i = APSInt(128, false);
@@ -171,7 +186,7 @@ class FloatEval {
171186
a.convert(sem, rm, &cvt_exact);
172187
break;
173188
case OpCode::FToDoubleToF:
174-
a.convert(APFloat::IEEEsingle(), rm, &cvt_exact);
189+
a.convert(APFloat::IEEEdouble(), rm, &cvt_exact);
175190
a.convert(sem, rm, &cvt_exact);
176191
break;
177192
default:
@@ -183,6 +198,24 @@ class FloatEval {
183198
return (StatusFlags)status;
184199
}
185200

201+
/** Evaluate a dynamically specified operation with the given configuration */
202+
/* FIXME(perf): since some of these may allocate, we should have an init
203+
* function to prealloc, then pass a pointer to this call. */
204+
static StatusFlags eval(OpCode op, Round round, UInt ai, UInt bi,
205+
UInt ci, UInt &out) {
206+
#if __cpp_exceptions
207+
try {
208+
return eval_impl(op, round, ai, bi, ci, out);
209+
} catch (const std::exception& e) {
210+
return handle_std_exception(e);
211+
} catch(...) {
212+
return handle_unknown_exception();
213+
}
214+
#else
215+
return eval_impl(op, round, ai, bi, ci, out);
216+
#endif
217+
}
218+
186219
static const fltSemantics &getSemantics() {
187220
return APFloat::EnumToSemantics(S);
188221
}
@@ -206,8 +239,13 @@ class EvalX87F80: public FloatEval<APFloat::Semantics::S_x87DoubleExtended, uint
206239
return Ty::eval(op, round, ai, bi, ci, out); \
207240
}
208241

242+
/* NB: Every symbol defined here also needs to be in the list in build.rs, otherwise they
243+
* will get pruned during optimization. */
209244
extern "C" {
210-
/* NB: Every symbol defined here also needs to be in the list in build.rs */
245+
const char *check_error() {
246+
return AP_ERROR;
247+
}
248+
211249
MAKE_EXTERN(EvalBrainF16, cxx_apf_eval_op_brainf16);
212250
MAKE_EXTERN(EvalIeee16, cxx_apf_eval_op_ieee16);
213251
MAKE_EXTERN(EvalIeee32, cxx_apf_eval_op_ieee32);

‎fuzz/src/apf_fuzz.rs‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,70 @@ impl<T> FuzzOp<T> {
7373
}
7474
}
7575

76+
/// A testable operation, which can be encoded as a byte.
77+
#[derive(Copy, Clone, Debug, PartialEq)]
78+
pub enum Op {
79+
Neg = 0,
80+
Add = 1,
81+
Sub = 2,
82+
Mul = 3,
83+
Div = 4,
84+
Rem = 5,
85+
MulAdd = 6,
86+
FToI128ToF = 7,
87+
FToU128ToF = 8,
88+
FToSingleToF = 9,
89+
FToDoubleToF = 10,
90+
}
91+
92+
impl Op {
93+
pub fn from_u8(tag: u8) -> Option<Self> {
94+
let v = match tag {
95+
x if x == Self::Neg.to_u8() => Self::Neg,
96+
x if x == Self::Add.to_u8() => Self::Add,
97+
x if x == Self::Sub.to_u8() => Self::Sub,
98+
x if x == Self::Mul.to_u8() => Self::Mul,
99+
x if x == Self::Div.to_u8() => Self::Div,
100+
x if x == Self::Rem.to_u8() => Self::Rem,
101+
x if x == Self::MulAdd.to_u8() => Self::MulAdd,
102+
x if x == Self::FToI128ToF.to_u8() => Self::FToI128ToF,
103+
x if x == Self::FToU128ToF.to_u8() => Self::FToU128ToF,
104+
x if x == Self::FToSingleToF.to_u8() => Self::FToSingleToF,
105+
x if x == Self::FToDoubleToF.to_u8() => Self::FToDoubleToF,
106+
_ => return None,
107+
};
108+
Some(v)
109+
}
110+
111+
pub fn to_u8(self) -> u8 {
112+
self as u8
113+
}
114+
115+
pub fn airity(self) -> Arity {
116+
match self {
117+
Op::Neg => Arity::Unary,
118+
Op::Add => Arity::Binary,
119+
Op::Sub => Arity::Binary,
120+
Op::Mul => Arity::Binary,
121+
Op::Div => Arity::Binary,
122+
Op::Rem => Arity::Binary,
123+
Op::MulAdd => Arity::Ternary,
124+
Op::FToI128ToF => Arity::Unary,
125+
Op::FToU128ToF => Arity::Unary,
126+
Op::FToSingleToF => Arity::Unary,
127+
Op::FToDoubleToF => Arity::Unary,
128+
}
129+
}
130+
}
131+
132+
/// Number of inputs to an operation.
133+
#[derive(Copy, Clone, Debug)]
134+
pub enum Arity {
135+
Unary = 1,
136+
Binary = 2,
137+
Ternary = 3,
138+
}
139+
76140
impl<HF> FuzzOp<HF>
77141
where
78142
HF: num_traits::Float

0 commit comments

Comments
 (0)