| Message ID | 20201015134137.205958-1-alxndr@bu.edu |
|---|---|
| Headers | show |
| Series | Add a General Virtual Device Fuzzer | expand |
Patchew URL: https://patchew.org/QEMU/20201015134137.205958-1-alxndr@bu.edu/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20201015134137.205958-1-alxndr@bu.edu
Subject: [PATCH v4 00/16] Add a General Virtual Device Fuzzer
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
- [tag update] patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com -> patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com
* [new tag] patchew/20201015134137.205958-1-alxndr@bu.edu -> patchew/20201015134137.205958-1-alxndr@bu.edu
Switched to a new branch 'test'
03323c2 scripts/oss-fuzz: remove the general-fuzz target
3c59bd5 fuzz: register predefined general-fuzz configs
0d80db3 fuzz: add general-fuzz configs for oss-fuzz
cea76d4 fuzz: add an "opaque" to the FuzzTarget struct
10cf373 fuzz: Add instructions for using general-fuzz
540fed2 scripts/oss-fuzz: Add crash trace minimization script
0550db0 scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
106e830 fuzz: add a crossover function to generic-fuzzer
90837f5 fuzz: add a DISABLE_PCI op to general-fuzzer
adf044f fuzz: Add support for custom crossover functions
0851cb4 fuzz: Add fuzzer callbacks to DMA-read functions
4acf44b fuzz: Declare DMA Read callback function
86fec34 fuzz: Add DMA support to the generic-fuzzer
237e9d6 fuzz: Add PCI features to the general fuzzer
41e6d7b fuzz: Add general virtual-device fuzzer
ba68f35 memory: Add FlatView foreach function
=== OUTPUT BEGIN ===
1/16 Checking commit ba68f35ed358 (memory: Add FlatView foreach function)
2/16 Checking commit 41e6d7bb40d3 (fuzz: Add general virtual-device fuzzer)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#19:
new file mode 100644
WARNING: line over 80 characters
#322: FILE: tests/qtest/fuzz/general_fuzz.c:299:
+ * Some commands can be variable-width, so we use a separator, SEPARATOR, to specify
WARNING: line over 80 characters
#330: FILE: tests/qtest/fuzz/general_fuzz.c:307:
+ * Simply, by removing the first byte, we end up with a very different sequence:
total: 0 errors, 3 warnings, 516 lines checked
Patch 2/16 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/16 Checking commit 237e9d6e734d (fuzz: Add PCI features to the general fuzzer)
4/16 Checking commit 86fec343b37a (fuzz: Add DMA support to the generic-fuzzer)
ERROR: externs should be avoided in .c files
#84: FILE: tests/qtest/fuzz/general_fuzz.c:120:
+void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write);
WARNING: Block comments use a leading /* on a separate line
#109: FILE: tests/qtest/fuzz/general_fuzz.c:145:
+ /* Regions are assumed to support 1-4 byte accesses unless
WARNING: Block comments use * on subsequent lines
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+ /* Regions are assumed to support 1-4 byte accesses unless
+ otherwise specified. */
WARNING: Block comments use a trailing */ on a separate line
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+ otherwise specified. */
total: 1 errors, 3 warnings, 299 lines checked
Patch 4/16 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/16 Checking commit 4acf44ba5630 (fuzz: Declare DMA Read callback function)
6/16 Checking commit 0851cb4b787d (fuzz: Add fuzzer callbacks to DMA-read functions)
7/16 Checking commit adf044f2b9d2 (fuzz: Add support for custom crossover functions)
8/16 Checking commit 90837f5a8315 (fuzz: add a DISABLE_PCI op to general-fuzzer)
9/16 Checking commit 106e83015d43 (fuzz: add a crossover function to generic-fuzzer)
10/16 Checking commit 0550db09152f (scripts/oss-fuzz: Add script to reorder a general-fuzzer trace)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#22:
new file mode 100755
WARNING: line over 80 characters
#117: FILE: scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py:91:
+ sys.stderr.write("Warning: Likely double fetch on line {}.\n"
total: 0 errors, 2 warnings, 103 lines checked
Patch 10/16 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/16 Checking commit 540fed26dc40 (scripts/oss-fuzz: Add crash trace minimization script)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#17:
new file mode 100755
total: 0 errors, 1 warnings, 157 lines checked
Patch 11/16 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/16 Checking commit 10cf3731768d (fuzz: Add instructions for using general-fuzz)
13/16 Checking commit cea76d423400 (fuzz: add an "opaque" to the FuzzTarget struct)
14/16 Checking commit 0d80db351b77 (fuzz: add general-fuzz configs for oss-fuzz)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#15:
new file mode 100644
total: 0 errors, 1 warnings, 172 lines checked
Patch 14/16 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
15/16 Checking commit 3c59bd5b67df (fuzz: register predefined general-fuzz configs)
16/16 Checking commit 03323c23616b (scripts/oss-fuzz: remove the general-fuzz target)
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20201015134137.205958-1-alxndr@bu.edu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
If you have to do a v5, the correct word is generic not general. :) Otherwise looks great. Paolo On 15/10/20 15:41, Alexander Bulekov wrote: > v4: > - Replace yaml + c template-based oss-fuzz configs, with C code to > register a FuzzTarget for each config (as suggested by Paolo) > - Replicate the functionality of address_space_write_rom to ensure > matching behavior when QTEST_LOG is enabled > - Improve code documentation/comments > - Small formatting changes > v3: > - Use flatviews to help select regions for fuzzing > - Meson-related changes > - Add some documentation > - Improve minimalization script to trim write{bwlq} commands > v2: > - Remove QOS dependency. > - Add a custom crossover function > - Fix broken minimization scripts > - Fixes to the IO region and DMA handling code > > This is a general virtual-device fuzzer, designed to fuzz devices over Port IO, > MMIO, and DMA. > > To get started with this: > 1. Build the fuzzers (see docs/devel/fuzzing.txt) > Note: Build with --enable-sanitizers, or create a "dictionary file": > echo kw1=\"FUZZ\" > dict > and pass it as an argument to libFuzzer with -dict=./dict > This magic value is a command separator that lets the fuzzer perform > multiple IO actions with a single input. > > 2. Pick the qemu arguments you wish to fuzz: > export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon" > > 3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the > "info qom-tree", "info qtree" and "info mtree" commands useful for identifying > these. Supports globbing. Here I will try to simultaneously fuzz(for no good > reason) virtio-balloon and e1000e, which is included by default in the q35: > export QEMU_FUZZ_OBJECTS='virtio* e1000*' > You can also try to fuzz the whole machine: > export QEMU_FUZZ_OBJECTS='*' > > 4. Run the fuzzer for 0 inputs. The fuzzer should output a list of > MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your > expectations. > ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0 > > 5. Run the fuzzer: > ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz > > > Basically, at the core, this fuzzer is an interpreter that splits the input > into a series of commands, such as mmio_write, pio_write, etc. We structure > these commands to hit only MemoryRegions that are associated with the devices > specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to > functions that are typically used by virtual-devices to read from RAM (DMA). > These hooks attempt to populate these DMA regions with fuzzed data, just in > time. > > Some of the issues I have found or reproduced with this fuzzer: > https://bugs.launchpad.net/bugs/1525123 > https://bugs.launchpad.net/bugs/1681439 > https://bugs.launchpad.net/bugs/1777315 > https://bugs.launchpad.net/bugs/1878034 > https://bugs.launchpad.net/bugs/1878043 > https://bugs.launchpad.net/bugs/1878054 > https://bugs.launchpad.net/bugs/1878057 > https://bugs.launchpad.net/bugs/1878067 > https://bugs.launchpad.net/bugs/1878134 > https://bugs.launchpad.net/bugs/1878136 > https://bugs.launchpad.net/bugs/1878253 > https://bugs.launchpad.net/bugs/1878255 > https://bugs.launchpad.net/bugs/1878259 > https://bugs.launchpad.net/bugs/1878263 > https://bugs.launchpad.net/bugs/1878323 > https://bugs.launchpad.net/bugs/1878641 > https://bugs.launchpad.net/bugs/1878642 > https://bugs.launchpad.net/bugs/1878645 > https://bugs.launchpad.net/bugs/1878651 > https://bugs.launchpad.net/bugs/1879223 > https://bugs.launchpad.net/bugs/1879227 > https://bugs.launchpad.net/bugs/1879531 > https://bugs.launchpad.net/bugs/1880355 > https://bugs.launchpad.net/bugs/1880539 > https://bugs.launchpad.net/bugs/1884693 > https://bugs.launchpad.net/bugs/1886362 > https://bugs.launchpad.net/bugs/1887303 > https://bugs.launchpad.net/bugs/1887309 > https://bugs.launchpad.net/bugs/697510 > > Alexander Bulekov (16): > memory: Add FlatView foreach function > fuzz: Add general virtual-device fuzzer > fuzz: Add PCI features to the general fuzzer > fuzz: Add DMA support to the generic-fuzzer > fuzz: Declare DMA Read callback function > fuzz: Add fuzzer callbacks to DMA-read functions > fuzz: Add support for custom crossover functions > fuzz: add a DISABLE_PCI op to general-fuzzer > fuzz: add a crossover function to generic-fuzzer > scripts/oss-fuzz: Add script to reorder a general-fuzzer trace > scripts/oss-fuzz: Add crash trace minimization script > fuzz: Add instructions for using general-fuzz > fuzz: add an "opaque" to the FuzzTarget struct > fuzz: add general-fuzz configs for oss-fuzz > fuzz: register predefined general-fuzz configs > scripts/oss-fuzz: remove the general-fuzz target > > docs/devel/fuzzing.txt | 39 + > include/exec/memory.h | 21 + > include/exec/memory_ldst_cached.h.inc | 3 + > memory_ldst.c.inc | 4 + > scripts/oss-fuzz/build.sh | 6 + > scripts/oss-fuzz/minimize_qtest_trace.py | 157 +++ > .../oss-fuzz/reorder_fuzzer_qtest_trace.py | 103 ++ > softmmu/memory.c | 23 + > softmmu/physmem.c | 2 + > tests/qtest/fuzz/fuzz.c | 13 + > tests/qtest/fuzz/fuzz.h | 28 + > tests/qtest/fuzz/general_fuzz.c | 948 ++++++++++++++++++ > tests/qtest/fuzz/general_fuzz_configs.c | 140 +++ > tests/qtest/fuzz/general_fuzz_configs.h | 24 + > tests/qtest/fuzz/meson.build | 1 + > 15 files changed, 1512 insertions(+) > create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py > create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py > create mode 100644 tests/qtest/fuzz/general_fuzz.c > create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c > create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h >
v4: - Replace yaml + c template-based oss-fuzz configs, with C code to register a FuzzTarget for each config (as suggested by Paolo) - Replicate the functionality of address_space_write_rom to ensure matching behavior when QTEST_LOG is enabled - Improve code documentation/comments - Small formatting changes v3: - Use flatviews to help select regions for fuzzing - Meson-related changes - Add some documentation - Improve minimalization script to trim write{bwlq} commands v2: - Remove QOS dependency. - Add a custom crossover function - Fix broken minimization scripts - Fixes to the IO region and DMA handling code This is a general virtual-device fuzzer, designed to fuzz devices over Port IO, MMIO, and DMA. To get started with this: 1. Build the fuzzers (see docs/devel/fuzzing.txt) Note: Build with --enable-sanitizers, or create a "dictionary file": echo kw1=\"FUZZ\" > dict and pass it as an argument to libFuzzer with -dict=./dict This magic value is a command separator that lets the fuzzer perform multiple IO actions with a single input. 2. Pick the qemu arguments you wish to fuzz: export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon" 3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the "info qom-tree", "info qtree" and "info mtree" commands useful for identifying these. Supports globbing. Here I will try to simultaneously fuzz(for no good reason) virtio-balloon and e1000e, which is included by default in the q35: export QEMU_FUZZ_OBJECTS='virtio* e1000*' You can also try to fuzz the whole machine: export QEMU_FUZZ_OBJECTS='*' 4. Run the fuzzer for 0 inputs. The fuzzer should output a list of MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your expectations. ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0 5. Run the fuzzer: ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz Basically, at the core, this fuzzer is an interpreter that splits the input into a series of commands, such as mmio_write, pio_write, etc. We structure these commands to hit only MemoryRegions that are associated with the devices specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to functions that are typically used by virtual-devices to read from RAM (DMA). These hooks attempt to populate these DMA regions with fuzzed data, just in time. Some of the issues I have found or reproduced with this fuzzer: https://bugs.launchpad.net/bugs/1525123 https://bugs.launchpad.net/bugs/1681439 https://bugs.launchpad.net/bugs/1777315 https://bugs.launchpad.net/bugs/1878034 https://bugs.launchpad.net/bugs/1878043 https://bugs.launchpad.net/bugs/1878054 https://bugs.launchpad.net/bugs/1878057 https://bugs.launchpad.net/bugs/1878067 https://bugs.launchpad.net/bugs/1878134 https://bugs.launchpad.net/bugs/1878136 https://bugs.launchpad.net/bugs/1878253 https://bugs.launchpad.net/bugs/1878255 https://bugs.launchpad.net/bugs/1878259 https://bugs.launchpad.net/bugs/1878263 https://bugs.launchpad.net/bugs/1878323 https://bugs.launchpad.net/bugs/1878641 https://bugs.launchpad.net/bugs/1878642 https://bugs.launchpad.net/bugs/1878645 https://bugs.launchpad.net/bugs/1878651 https://bugs.launchpad.net/bugs/1879223 https://bugs.launchpad.net/bugs/1879227 https://bugs.launchpad.net/bugs/1879531 https://bugs.launchpad.net/bugs/1880355 https://bugs.launchpad.net/bugs/1880539 https://bugs.launchpad.net/bugs/1884693 https://bugs.launchpad.net/bugs/1886362 https://bugs.launchpad.net/bugs/1887303 https://bugs.launchpad.net/bugs/1887309 https://bugs.launchpad.net/bugs/697510 Alexander Bulekov (16): memory: Add FlatView foreach function fuzz: Add general virtual-device fuzzer fuzz: Add PCI features to the general fuzzer fuzz: Add DMA support to the generic-fuzzer fuzz: Declare DMA Read callback function fuzz: Add fuzzer callbacks to DMA-read functions fuzz: Add support for custom crossover functions fuzz: add a DISABLE_PCI op to general-fuzzer fuzz: add a crossover function to generic-fuzzer scripts/oss-fuzz: Add script to reorder a general-fuzzer trace scripts/oss-fuzz: Add crash trace minimization script fuzz: Add instructions for using general-fuzz fuzz: add an "opaque" to the FuzzTarget struct fuzz: add general-fuzz configs for oss-fuzz fuzz: register predefined general-fuzz configs scripts/oss-fuzz: remove the general-fuzz target docs/devel/fuzzing.txt | 39 + include/exec/memory.h | 21 + include/exec/memory_ldst_cached.h.inc | 3 + memory_ldst.c.inc | 4 + scripts/oss-fuzz/build.sh | 6 + scripts/oss-fuzz/minimize_qtest_trace.py | 157 +++ .../oss-fuzz/reorder_fuzzer_qtest_trace.py | 103 ++ softmmu/memory.c | 23 + softmmu/physmem.c | 2 + tests/qtest/fuzz/fuzz.c | 13 + tests/qtest/fuzz/fuzz.h | 28 + tests/qtest/fuzz/general_fuzz.c | 948 ++++++++++++++++++ tests/qtest/fuzz/general_fuzz_configs.c | 140 +++ tests/qtest/fuzz/general_fuzz_configs.h | 24 + tests/qtest/fuzz/meson.build | 1 + 15 files changed, 1512 insertions(+) create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py create mode 100644 tests/qtest/fuzz/general_fuzz.c create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h