Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provide for cargo afl, but it is not complete. #352

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/.cspell/project-dictionary.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,10 @@ winapi
xargo
Xdemangler
xtask
addseeds
cmin
gotcpu
showmap
tmin
whatsup
LOOPCOUNT
26 changes: 26 additions & 0 deletions docs/cargo-llvm-cov-afl.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
cargo-llvm-cov-afl

Use cargo llvm-cov afl to fuzz test your Rust projects with American Fuzzy Lop (AFL), a powerful fuzz testing tool that automatically finds bugs by feeding unexpected inputs to your programs.

SUBCOMMANDS:
addseeds Invoke afl-addseeds
analyze Invoke afl-analyze
cmin Invoke afl-cmin
config Build or rebuild AFL++
fuzz Invoke afl-fuzz
gotcpu Invoke afl-gotcpu
plot Invoke afl-plot
showmap Invoke afl-showmap
system-config Invoke afl-system-config (beware, called with sudo!)
tmin Invoke afl-tmin
whatsup Invoke afl-whatsup
help Print this message or the help of the given subcommand(s)

Arguments:
[ARGS]...

Options:
-h, --help Print help
-V, --version Print version

In addition to the subcommands above, Cargo subcommands are also supported (see `cargo help` for a list of all Cargo subcommands).
7 changes: 7 additions & 0 deletions src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -947,6 +947,9 @@ pub(crate) enum Subcommand {
/// Build and archive tests with cargo nextest
NextestArchive,

/// Run afl with cargo nextest
Afl,

// internal (unstable)
Demangle,
}
Expand All @@ -960,6 +963,7 @@ static CARGO_LLVM_COV_SHOW_ENV_USAGE: &str = include_str!("../docs/cargo-llvm-co
static CARGO_LLVM_COV_NEXTEST_USAGE: &str = include_str!("../docs/cargo-llvm-cov-nextest.txt");
static CARGO_LLVM_COV_NEXTEST_ARCHIVE_USAGE: &str =
include_str!("../docs/cargo-llvm-cov-nextest-archive.txt");
static CARGO_LLVM_COV_AFL_USAGE: &str = include_str!("../docs/cargo-llvm-cov-afl.txt");

impl Subcommand {
fn can_passthrough(subcommand: Self) -> bool {
Expand All @@ -976,6 +980,7 @@ impl Subcommand {
Self::ShowEnv => CARGO_LLVM_COV_SHOW_ENV_USAGE,
Self::Nextest { .. } => CARGO_LLVM_COV_NEXTEST_USAGE,
Self::NextestArchive => CARGO_LLVM_COV_NEXTEST_ARCHIVE_USAGE,
Self::Afl => CARGO_LLVM_COV_AFL_USAGE,
Self::Demangle => "", // internal API
}
}
Expand All @@ -990,6 +995,7 @@ impl Subcommand {
Self::ShowEnv => "show-env",
Self::Nextest { .. } => "nextest",
Self::NextestArchive => "nextest-archive",
Self::Afl => "afl",
Self::Demangle => "demangle",
}
}
Expand All @@ -1011,6 +1017,7 @@ impl FromStr for Subcommand {
"show-env" => Ok(Self::ShowEnv),
"nextest" => Ok(Self::Nextest { archive_file: false }),
"nextest-archive" => Ok(Self::NextestArchive),
"afl" => Ok(Self::Afl),
"demangle" => Ok(Self::Demangle),
_ => bail!("unrecognized subcommand {s}"),
}
Expand Down
66 changes: 66 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,15 @@ fn try_main() -> Result<()> {
create_dirs(cx)?;
archive_nextest(cx)?;
}
Subcommand::Afl => {
let cx = &Context::new(args)?;
clean::clean_partial(cx)?;
create_dirs(cx)?;
run_afl(cx)?;
if !cx.args.cov.no_report {
generate_report(cx)?;
}
}
Subcommand::None | Subcommand::Test => {
let cx = &Context::new(args)?;
clean::clean_partial(cx)?;
Expand Down Expand Up @@ -498,6 +507,63 @@ fn run_run(cx: &Context) -> Result<()> {
Ok(())
}

fn run_afl(cx: &Context) -> Result<()> {
let mut cargo = cx.cargo();

set_env(cx, &mut cargo, IsNextest(false))?;

let input_dir = "inputs/";
let output_dir = "outputs/";

fs::create_dir_all(input_dir)?;
fs::create_dir_all(output_dir)?;

if !cx.args.ignore_run_fail {
{
let mut cargo = cargo.clone();

cargo.arg("afl").arg("build");
cargo.arg("--target-dir").arg(cx.ws.target_dir.as_str());
Comment on lines +525 to +526
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to propagate flags passed by the user, such as releases. We probably need to create a new function like test_or_run_args in cargo.rs.

if term::verbose() {
status!("Running", "{cargo}");
cargo.stdout_to_stderr().run()?;
} else {
// Capture output to prevent duplicate warnings from appearing in two runs.
cargo.run_with_output()?;
}
}

// prepare seeds
let seeds = vec!["example input 1", "sample data 2", "test case 3"];

for (index, seed) in seeds.into_iter().enumerate() {
let file_path = format!("{input_dir}seed_{index}.txt");
let path = Path::new(&file_path);
let mut file = fs::File::create(path)?;
file.write_all(seed.as_bytes())?;
}
Comment on lines +536 to +544
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I beliave these should not be defined on our end, but should respect what the user passes as arguments.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I would fix it,


// set afl's loop count to 1000 to avoid AFL loop infinitely!
// AFL maybe exits a process several minutes later and produce a cov file.
cargo.set("AFL_FUZZER_LOOPCOUNT", "20")?;
cargo.arg("afl").arg("fuzz");
cargo.arg("-V").arg("10");
// set in and out
cargo.arg("-i").arg(input_dir);
cargo.arg("-o").arg(output_dir);
Comment on lines +546 to +553
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These probably have the same problem. Even if we define our own defaults, we need to respect them if they are specified by the user. Personally, I would prefer to report an error if an environment variable or argument is not set than to define our own on our end.

// set the executable
let bin_file = cx.ws.target_dir.join("debug").join(cx.ws.name.clone()).to_string();
cargo.arg(bin_file.as_str());

if term::verbose() {
status!("Running", "{cargo}");
}
stdout_to_stderr(cx, &mut cargo);
cargo.run()?;
}
Ok(())
}

fn stdout_to_stderr(cx: &Context, cargo: &mut ProcessBuilder) {
if cx.args.cov.no_report
|| cx.args.cov.output_dir.is_some()
Expand Down