| Message ID | 20200922104932.46384-1-mreitz@redhat.com |
|---|---|
| Headers | show |
| Series | block/export: Allow exporting BDSs via FUSE | expand |
On Tue, Sep 22, 2020 at 12:49:12PM +0200, Max Reitz wrote: > Based-on: <20200907182011.521007-1-kwolf@redhat.com> > (“block/export: Add infrastructure and QAPI for block exports”) > > (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) > > v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html > > Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 > Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 > > > Hi, > > Ever since I found out that you can mount FUSE filesystems on regular > files (not just directories), I had the idea of adding FUSE block > exports to qemu where you can export block nodes as raw images. The > best thing is that you’d be able to mount an image on itself, so > whatever format it may be in, qemu lets it appear as a raw image (and > you can then use regular tools like dd on it). > > The performance is quite bad so far, but we can always try to improve it > if the need arises. For now I consider it mostly a cute feature to get > easy access to the raw contents of image files in any format (without > requiring root rights). Aside from the iotests, so you forsee any particular use cases where this feature is desirable / important ? Looking at it from a security POV, I'm not thrilled about the idea of granting QEMU permission to use the mount syscall for seccomp or SELinux. IOW, I expect this feature won't be something we want to expose in QEMU guests managed by libvirt, which would limit how widely it can be used. QEMU can export NBD. Would it make sense to do this as an NBD client ? There's already https://libguestfs.org/nbdfuse.1.html but IIUC that exposes it as a file within a dir. Presumably it is not too hard to make it support exposing it directly as a file too. I wonder how performance compares between your native FUSE impl in QEMU vs NBD FUSE ? Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 22.09.20 17:58, Daniel P. Berrangé wrote: > On Tue, Sep 22, 2020 at 12:49:12PM +0200, Max Reitz wrote: >> Based-on: <20200907182011.521007-1-kwolf@redhat.com> >> (“block/export: Add infrastructure and QAPI for block exports”) >> >> (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) >> >> v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html >> >> Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 >> Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 >> >> >> Hi, >> >> Ever since I found out that you can mount FUSE filesystems on regular >> files (not just directories), I had the idea of adding FUSE block >> exports to qemu where you can export block nodes as raw images. The >> best thing is that you’d be able to mount an image on itself, so >> whatever format it may be in, qemu lets it appear as a raw image (and >> you can then use regular tools like dd on it). >> >> The performance is quite bad so far, but we can always try to improve it >> if the need arises. For now I consider it mostly a cute feature to get >> easy access to the raw contents of image files in any format (without >> requiring root rights). > > Aside from the iotests, so you forsee any particular use cases > where this feature is desirable / important ? No. I implemented this feature for fun last year (when I realized that FUSE allows regular files to be mount points), and I got positive reactions. I assumed others would find it as nice as me to be able to quickly access an image file without requiring root rights (and then device file accesses), or setting up an NBD chain. (Though it should be noted that when I first came up with the feature, nbdfuse did not exist yet.) (It should also be noted that my original idea was to have a new executable qemu-blkfuse that would basically allow you to invoke “qemu-blkfuse $img”, and then $img would appear as a raw image. To me, that appeared very useful because it was so simple. I admit that the current proposal, which relies on the storage-daemon, has none of that simplicity. But if that’s the problem that prevents this from being considered useful, I’m sure we (I) can figure something out. Perhaps a simple script, bundled with qemu, that can generate -blockdev invocations based on the result of file(1).) > Looking at it from a security POV, I'm not thrilled about the > idea of granting QEMU permission to use the mount syscall for > seccomp or SELinux. IOW, I expect this feature won't be something > we want to expose in QEMU guests managed by libvirt, which would > limit how widely it can be used. I don’t expect this to be used through QEMU very much, but through the storage daemon. I assume that for the storage daemon, the permissions can effectively be fine-tuned for each export, because you can “just” launch another instance. > QEMU can export NBD. Would it make sense to do this as an NBD > client ? There's already https://libguestfs.org/nbdfuse.1.html > but IIUC that exposes it as a file within a dir. Presumably > it is not too hard to make it support exposing it directly as > a file too. I don’t like that idea very much, because my main gripe with the current state of my proposal is that it’s more cumbersome than “qemu-blkfuse $img”. Adding more indirections won’t make it simpler. > I wonder how performance compares between your native FUSE > impl in QEMU vs NBD FUSE ? Last year, I tried various ways of improving performance and nothing really amounted to much. So in the end I settled for a simple and naive implementation, for it to be improved in case anyone cares for it. Max
On Tue, Sep 22, 2020 at 04:58:38PM +0100, Daniel P. Berrangé wrote: > On Tue, Sep 22, 2020 at 12:49:12PM +0200, Max Reitz wrote: > > Based-on: <20200907182011.521007-1-kwolf@redhat.com> > > (“block/export: Add infrastructure and QAPI for block exports”) > > > > (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) > > > > v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html > > > > Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 > > Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 > > > > > > Hi, > > > > Ever since I found out that you can mount FUSE filesystems on regular > > files (not just directories), I had the idea of adding FUSE block > > exports to qemu where you can export block nodes as raw images. The > > best thing is that you’d be able to mount an image on itself, so > > whatever format it may be in, qemu lets it appear as a raw image (and > > you can then use regular tools like dd on it). > > > > The performance is quite bad so far, but we can always try to improve it > > if the need arises. For now I consider it mostly a cute feature to get > > easy access to the raw contents of image files in any format (without > > requiring root rights). > > Aside from the iotests, so you forsee any particular use cases > where this feature is desirable / important ? Alice Frosi is working on a qemu-storage-daemon-based project where the FUSE export type is useful. In this case qemu-storage-daemon is used stand-alone without a guest or libvirt directly involved. The goal is just to export disk images and how they are consumed is the user's responsibility (processes, containers, guests). > Looking at it from a security POV, I'm not thrilled about the > idea of granting QEMU permission to use the mount syscall for > seccomp or SELinux. IOW, I expect this feature won't be something > we want to expose in QEMU guests managed by libvirt, which would > limit how widely it can be used. I have CCed Miklos Szeredi, the Linux FUSE maintainer, to check what the options are for unprivileged mounting of a FUSE file system: 1. libfuse invokes open("/dev/fuse") + mount() 2. libfuse spawns the fusermount3 suid root helper 3. Any other options? D-Bus? etc > QEMU can export NBD. Would it make sense to do this as an NBD > client ? There's already https://libguestfs.org/nbdfuse.1.html > but IIUC that exposes it as a file within a dir. Presumably > it is not too hard to make it support exposing it directly as > a file too. > > I wonder how performance compares between your native FUSE > impl in QEMU vs NBD FUSE ? NBD exports are useful for networks but shouldn't preclude other export types from being merged just because they can be implemented on top of NBD. Native export types are simpler to manage and have less performance overhead than stacking additional tools on top of NBD. Stefan
Am 22.09.2020 um 12:49 hat Max Reitz geschrieben: > Based-on: <20200907182011.521007-1-kwolf@redhat.com> > (“block/export: Add infrastructure and QAPI for block exports”) > > (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) > > v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html > > Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 > Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 > > > Hi, > > Ever since I found out that you can mount FUSE filesystems on regular > files (not just directories), I had the idea of adding FUSE block > exports to qemu where you can export block nodes as raw images. The > best thing is that you’d be able to mount an image on itself, so > whatever format it may be in, qemu lets it appear as a raw image (and > you can then use regular tools like dd on it). > > The performance is quite bad so far, but we can always try to improve it > if the need arises. For now I consider it mostly a cute feature to get > easy access to the raw contents of image files in any format (without > requiring root rights). > > In this version (as opposed to v1 linked above), I integrated the FUSE > export code into Kevin’s proposed common infrastructure for block > exports. Patches 5-16, 19 and 20: Reviewed-by: Kevin Wolf <kwolf@redhat.com>
On 15.10.20 14:01, Kevin Wolf wrote: > Am 22.09.2020 um 12:49 hat Max Reitz geschrieben: >> Based-on: <20200907182011.521007-1-kwolf@redhat.com> >> (“block/export: Add infrastructure and QAPI for block exports”) >> >> (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) >> >> v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html >> >> Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 >> Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 >> >> >> Hi, >> >> Ever since I found out that you can mount FUSE filesystems on regular >> files (not just directories), I had the idea of adding FUSE block >> exports to qemu where you can export block nodes as raw images. The >> best thing is that you’d be able to mount an image on itself, so >> whatever format it may be in, qemu lets it appear as a raw image (and >> you can then use regular tools like dd on it). >> >> The performance is quite bad so far, but we can always try to improve it >> if the need arises. For now I consider it mostly a cute feature to get >> easy access to the raw contents of image files in any format (without >> requiring root rights). >> >> In this version (as opposed to v1 linked above), I integrated the FUSE >> export code into Kevin’s proposed common infrastructure for block >> exports. > > Patches 5-16, 19 and 20: > > Reviewed-by: Kevin Wolf <kwolf@redhat.com> Thanks! So far, (just FYI,) I’ve tried to address your remarks, gathered them in a single fat commit, and pushed everything here: https://github.com/XanClic/qemu/tree/fuse-exports-next (There’s also something about two bugs in fuse_fallocate(), where there are these two I/O loops (one for blk_pwrite_zeroes(), one for blk_pdiscard()), where I both forgot to increment @offset (alongside decrementing @length). Oops.) (And I’ve renamed init_fuse() to init_exports_table(), because that got really confusing with fuse_init().) I’ll split that up, squash it into the respective patches, and send v3 when I get back from PTO. Max
Based-on: <20200907182011.521007-1-kwolf@redhat.com> (“block/export: Add infrastructure and QAPI for block exports”) (Though its patch 16 needs a s/= \&blk_exp_nbd/= drv/ on top.) v1: https://lists.nongnu.org/archive/html/qemu-block/2019-12/msg00451.html Branch: https://github.com/XanClic/qemu.git fuse-exports-v2 Branch: https://git.xanclic.moe/XanClic/qemu.git fuse-exports-v2 Hi, Ever since I found out that you can mount FUSE filesystems on regular files (not just directories), I had the idea of adding FUSE block exports to qemu where you can export block nodes as raw images. The best thing is that you’d be able to mount an image on itself, so whatever format it may be in, qemu lets it appear as a raw image (and you can then use regular tools like dd on it). The performance is quite bad so far, but we can always try to improve it if the need arises. For now I consider it mostly a cute feature to get easy access to the raw contents of image files in any format (without requiring root rights). In this version (as opposed to v1 linked above), I integrated the FUSE export code into Kevin’s proposed common infrastructure for block exports. This series does the following: First, add the FUSE export module (block/export/fuse.c) that implements the basic file access functions. (Note that you need libfuse 3.8.0 or later for SEEK_HOLE/SEEK_DATA.) Second, it allows using FUSE exports as a protocol in the iotests and makes many iotests work with it. (The file node is exported by a background qemu instance to $SOCK_DIR.) This gives us a lot of coverage for, well, not free (it does take twelve patches), but for cheap; but there are still some more specialized things we want to test, so third and last, this series adds an iotest dedicated to FUSE exports. (Note that as opposed to v1, I did run the iotests this time.) Some notable changes from v1: - Integrated everything into Kevin’s block-export infrastructure - Use the storage daemon instead of full QEMU to provide FUSE exports when running the iotests with -fuse - meson rebase - Some other rebase conflicts git-backport-diff against v1: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/20:[0007] [FC] 'configure: Detect libfuse' 002/20:[0255] [FC] 'fuse: Allow exporting BDSs via FUSE' 003/20:[0062] [FC] 'fuse: Implement standard FUSE operations' 004/20:[0018] [FC] 'fuse: Allow growable exports' 005/20:[0016] [FC] 'fuse: (Partially) implement fallocate()' 006/20:[0008] [FC] 'fuse: Implement hole detection through lseek' 007/20:[0036] [FC] 'iotests: Do not needlessly filter _make_test_img' 008/20:[----] [--] 'iotests: Do not pipe _make_test_img' 009/20:[0012] [FC] 'iotests: Use convert -n in some cases' 010/20:[0006] [FC] 'iotests: Avoid renaming images' 011/20:[0008] [FC] 'iotests: Derive image names from $TEST_IMG' 012/20:[----] [--] 'iotests/091: Use _cleanup_qemu instad of "wait"' 013/20:[0008] [FC] 'iotests: Restrict some Python tests to file' 014/20:[0010] [FC] 'iotests: Let _make_test_img guess $TEST_IMG_FILE' 015/20:[down] 'iotests/287: Clean up subshell test image' 016/20:[down] 'storage-daemon: Call bdrv_close_all() on exit' 017/20:[down] 'iotests: Give access to the qemu-storage-daemon' 018/20:[0042] [FC] 'iotests: Allow testing FUSE exports' 019/20:[0026] [FC] 'iotests: Enable fuse for many tests' 020/20:[0104] [FC] 'iotests/281: Add test for FUSE exports' Max Reitz (20): configure: Detect libfuse fuse: Allow exporting BDSs via FUSE fuse: Implement standard FUSE operations fuse: Allow growable exports fuse: (Partially) implement fallocate() fuse: Implement hole detection through lseek iotests: Do not needlessly filter _make_test_img iotests: Do not pipe _make_test_img iotests: Use convert -n in some cases iotests/046: Avoid renaming images iotests: Derive image names from $TEST_IMG iotests/091: Use _cleanup_qemu instad of "wait" iotests: Restrict some Python tests to file iotests: Let _make_test_img guess $TEST_IMG_FILE iotests/287: Clean up subshell test image storage-daemon: Call bdrv_close_all() on exit iotests: Give access to the qemu-storage-daemon iotests: Allow testing FUSE exports iotests: Enable fuse for many tests iotests/308: Add test for FUSE exports configure | 66 +++ qapi/block-export.json | 28 +- include/block/fuse.h | 30 ++ block.c | 1 + block/export/export.c | 4 + block/export/fuse.c | 645 +++++++++++++++++++++++++++ storage-daemon/qemu-storage-daemon.c | 3 + block/export/meson.build | 1 + meson.build | 7 + tests/qemu-iotests/025 | 2 +- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/028 | 16 +- tests/qemu-iotests/028.out | 3 + tests/qemu-iotests/031 | 2 +- tests/qemu-iotests/034 | 2 +- tests/qemu-iotests/036 | 2 +- tests/qemu-iotests/037 | 2 +- tests/qemu-iotests/038 | 2 +- tests/qemu-iotests/039 | 2 +- tests/qemu-iotests/046 | 7 +- tests/qemu-iotests/046.out | 2 +- tests/qemu-iotests/050 | 2 +- tests/qemu-iotests/054 | 2 +- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/071 | 21 +- tests/qemu-iotests/079 | 2 +- tests/qemu-iotests/080 | 2 +- tests/qemu-iotests/089 | 5 +- tests/qemu-iotests/089.out | 1 + tests/qemu-iotests/090 | 2 +- tests/qemu-iotests/091 | 5 +- tests/qemu-iotests/095 | 2 +- tests/qemu-iotests/097 | 2 +- tests/qemu-iotests/098 | 2 +- tests/qemu-iotests/102 | 2 +- tests/qemu-iotests/103 | 2 +- tests/qemu-iotests/106 | 2 +- tests/qemu-iotests/107 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/111 | 2 +- tests/qemu-iotests/112 | 2 +- tests/qemu-iotests/115 | 2 +- tests/qemu-iotests/117 | 2 +- tests/qemu-iotests/120 | 2 +- tests/qemu-iotests/121 | 2 +- tests/qemu-iotests/127 | 2 +- tests/qemu-iotests/133 | 2 +- tests/qemu-iotests/137 | 2 +- tests/qemu-iotests/138 | 2 +- tests/qemu-iotests/140 | 2 +- tests/qemu-iotests/154 | 2 +- tests/qemu-iotests/161 | 14 +- tests/qemu-iotests/171 | 2 +- tests/qemu-iotests/174 | 10 +- tests/qemu-iotests/175 | 8 +- tests/qemu-iotests/176 | 2 +- tests/qemu-iotests/177 | 2 +- tests/qemu-iotests/179 | 2 +- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/186 | 2 +- tests/qemu-iotests/187 | 2 +- tests/qemu-iotests/191 | 2 +- tests/qemu-iotests/195 | 2 +- tests/qemu-iotests/200 | 5 +- tests/qemu-iotests/200.out | 4 +- tests/qemu-iotests/204 | 2 +- tests/qemu-iotests/206 | 3 +- tests/qemu-iotests/214 | 2 +- tests/qemu-iotests/217 | 2 +- tests/qemu-iotests/220 | 2 +- tests/qemu-iotests/221 | 2 +- tests/qemu-iotests/229 | 5 +- tests/qemu-iotests/229.out | 6 +- tests/qemu-iotests/242 | 3 +- tests/qemu-iotests/247 | 2 +- tests/qemu-iotests/249 | 8 +- tests/qemu-iotests/250 | 2 +- tests/qemu-iotests/252 | 2 +- tests/qemu-iotests/265 | 2 +- tests/qemu-iotests/268 | 2 +- tests/qemu-iotests/272 | 2 +- tests/qemu-iotests/273 | 2 +- tests/qemu-iotests/279 | 2 +- tests/qemu-iotests/286 | 2 +- tests/qemu-iotests/287 | 6 +- tests/qemu-iotests/289 | 2 +- tests/qemu-iotests/290 | 2 +- tests/qemu-iotests/291 | 2 +- tests/qemu-iotests/292 | 2 +- tests/qemu-iotests/293 | 2 +- tests/qemu-iotests/294 | 2 +- tests/qemu-iotests/305 | 2 +- tests/qemu-iotests/308 | 339 ++++++++++++++ tests/qemu-iotests/308.out | 97 ++++ tests/qemu-iotests/check | 17 + tests/qemu-iotests/common.filter | 5 +- tests/qemu-iotests/common.rc | 181 +++++++- tests/qemu-iotests/group | 1 + 98 files changed, 1559 insertions(+), 126 deletions(-) create mode 100644 include/block/fuse.h create mode 100644 block/export/fuse.c create mode 100755 tests/qemu-iotests/308 create mode 100644 tests/qemu-iotests/308.out