Exclude synced test cases from transmission

This commit is contained in:
maride 2021-04-26 23:56:03 +02:00
parent 4ce7e049fb
commit 9e80ef9290
2 changed files with 24 additions and 1 deletions

View File

@ -8,6 +8,7 @@ Transfer AFL files over a mesh to fuzz across multiple servers
- Automatically syncs the main fuzzer to secondary nodes, and all secondary fuzzers back to the main node - Automatically syncs the main fuzzer to secondary nodes, and all secondary fuzzers back to the main node
- Encrypts traffic between nodes using AES-256, dropping plaintext packets - Encrypts traffic between nodes using AES-256, dropping plaintext packets
- Usable on UNIXoid (Linux, OSX) systems and Windows - Usable on UNIXoid (Linux, OSX) systems and Windows
- Reduces the amount of transmitted test cases to a bare minimum
## Usage ## Usage
@ -47,4 +48,13 @@ Please note that there might be some edge cases when you don't want that behavio
- you expect your fuzzers to give the same (file) name to different test cases, in which case *afl-transmit* would mistakenly assume that the file has the same *contents* and not only the same *name* - you expect your fuzzers to give the same (file) name to different test cases, in which case *afl-transmit* would mistakenly assume that the file has the same *contents* and not only the same *name*
- you don't care for traffic - you don't care for traffic
To avoid reducing the transmitted files, add `--no-duplicates=false` as argument. To avoid reducing the transmitted files by comparing filenames, add `--no-duplicates=false` as argument.
Also on default, *afl-transmit* tries to check if the queue of the observed fuzzers contain test cases which originated from another fuzzer instance.
In that case, the file name contains the keyword "sync" in it, and looks e.g. like this: `id:001815,time:0,orig:id:001805,sync:main,src:001794`
If it was copied from another fuzzer, it means that the file is already present in the fuzzer cluster, and can safely be skipped on those fuzzer instances which copied it.
Please note that this will produce false positives if the filename of your testcases contain `,sync:` for whatever reason.
To avoid reducing the transmitted files by filtering synced files out, add `--avoid-synced=false` as argument.
If you still have trouble paying the invoice for your ISP due to heavy traffic usage, try increasing the `--rescan` value, so files are transmitted less often.

View File

@ -14,9 +14,14 @@ import (
var noDuplicates bool var noDuplicates bool
// avoidSynced is the flag to set if files containing "sync" should be packed or not.
// Those files are from another fuzzer - no need to pack them twice.
var avoidSynced bool
// RegisterPackerFlags registers flags which are required by the packer // RegisterPackerFlags registers flags which are required by the packer
func RegisterPackerFlags() { func RegisterPackerFlags() {
flag.BoolVar(&noDuplicates, "no-duplicates", true, "Avoid transmitting the same file multiple times, e.g. because it is present in multiple fuzzer's queues") flag.BoolVar(&noDuplicates, "no-duplicates", true, "Avoid transmitting the same file multiple times, e.g. because it is present in multiple fuzzer's queues")
flag.BoolVar(&avoidSynced, "avoid-synced", true, "Avoid transmitting files containing the keyword 'sync', as they are from other fuzzers anyways, and should be included by their afl-transmit instance")
} }
// PackFuzzers packs all targeted fuzzers into a TAR - at least queue/, fuzz_bitmap, fuzzer_stats // PackFuzzers packs all targeted fuzzers into a TAR - at least queue/, fuzz_bitmap, fuzzer_stats
@ -113,6 +118,14 @@ func packQueueFiles(tarWriter *tar.Writer, absPath string, relPath string, pkgCo
continue continue
} }
// Check if we should care for the keyword 'sync' in file name
if avoidSynced && strings.Contains(f.Name(), ",sync:") {
// seems like this file was put into the queue of this fuzzer by syncing it from another fuzzer. We don't
// need to transmit it then, because the fuzzer which found that case will have the same file but without
// the keyword "sync" in it. Simply put, we avoid sending the same file multiple times with different names.
continue
}
// Pack into the archive // Pack into the archive
packSingleFile(tarWriter, absPath, relPath, fmt.Sprintf("queue%c%s", os.PathSeparator, f.Name()), false) packSingleFile(tarWriter, absPath, relPath, fmt.Sprintf("queue%c%s", os.PathSeparator, f.Name()), false)