| Age | Commit message (Collapse) | Author | Lines |
|---|
|
This adds the libfuse changes needed to support request timeouts.
A timeout may be set by the server in its init call. If a request is not
completed by the timeout, the connection will be aborted by the kernel.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
|
|
Signed-off-by: izxl007 <zeng.zheng@zte.com.cn>
|
|
So far it was disabled as the series was not complete
yet.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
We want to especially test with and without io-uring being
enabled. Ideally without modifying all tests - that is
what the env variable can be used for.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
This might be useful to optimize code paths. For example, with
io-uring the request buffer is valid until the request is replied
to, while without io-uring the request buffer is only valid
in current thread context.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
It will never be modified, should be const.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
The actual function to write to /dev/fuse is offloaded
into _fuse_send_msg()
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Needed by follow up commits. container_of is actually
just moved/consolidated to util.h.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Header/payload separation is part of the fuse-io-uring protocol
and might be later on for /dev/fuse legacy communication as well.
This is a preparation commit, for now fuse_ll_ops2 is unused.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Issue with previous code was that fuse_session_exit()
didn't wake up the semaphore in fuse_loop_mt.c.
Lock, semaphore and all uses of checking for "exited"
are now moved to struct fuse_session to have it
available for the signal handler.
This also removes internal fuse_session_reset() calls,
as that makes testing hard. From git history I also
don't see why it was added.
Closes: https://github.com/libfuse/libfuse/issues/997
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
These are several buf size fixes
1) Set bufsize when HAVE_SPLICE is not defined. Addresses
https://github.com/libfuse/libfuse/issues/1184
2) Check in the read retry condition for bufsize, i.e. the value
passed to read and not for the buf->mem_size. Using buf->mem_size
can be startup racy. Additionally we now also set bufsize on
allocation to avoid these races.
3) Allocation and value assigned needs to follow the pattern
bufsize = se->bufsize;
buf_alloc(bufsize, internal)
buf->mem_size = bufsize;
I.e. bufsize has to be retrieved first, as se->bufsize might
change at anytime - the value used for allocation and
must not differ from the value actually used.
This also tries to set large sizes in passthrough_hp, to catch
issues in xfstests - though requires to set
/proc/sys/fs/fuse/max_pages_limit
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Default is having usdt disabled.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
|
|
Add user statically-defined tracepoints for request lifecycle.
Verified by:
[machine]$ readelf -n /home/libfuse/build/lib/libfuse3.so | grep -A 1000 '.note.stapsdt'
Displaying notes found in: .note.stapsdt
Owner Data size Description
stapsdt 0x00000035 NT_STAPSDT (SystemTap probe descriptors)
Provider: libfuse
Name: request_receive
Location: 0x0000000000016360, Base: 0x00000000000002fc, Semaphore: 0x0000000000000000
Arguments: -4@%edi
stapsdt 0x00000055 NT_STAPSDT (SystemTap probe descriptors)
Provider: libfuse
Name: request_reply
Location: 0x0000000000017c0f, Base: 0x00000000000002fc, Semaphore: 0x0000000000000000
Arguments: 8@8(%r12) 4@(%r12) -4@4(%r12) -4@%r13d
stapsdt 0x0000003e NT_STAPSDT (SystemTap probe descriptors)
Provider: libfuse
Name: request_process
Location: 0x000000000001acf7, Base: 0x00000000000002fc, Semaphore: 0x0000000000000000
Arguments: 4@%r8d 4@%edx
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
|
|
32-bit conn->want flags been left to be ABI compatible to 3.10,
even though the so version was changed.
The more recent way is to use fuse_set_feature_flag(),
which will use conn->want_ext.
Given that we now have two flags (want and want_ext), we need
to convert and that brought several issues
- If the application sets conn->want, that needs to be set into
the lower 32 bit of conn->want_ext. As the application might
actually unset values, it really has to be a copy and not
just 'or' - fixed now.
- convert_to_conn_want_ext() actually needs to check for
_modified_ conn->want and conn->want_ext
- convert_to_conn_want_ext() must consider being called from
high and lowlevel interfact, with different want_ext_default
and want_default values. It is only a failure, if the application
changed both, conn->want and conn->want_ext. This function
was failing in issue #1171, because high level fuse_fs_init()
was changing values and then lowlevel do_init() was incorrectly
failing on that.
This also adds a new test (test_want_conversion) and sets
values into example/{hello.c,hello_ll.c}
Also some more internal users of conn->want are converted to
fuse_{set,unset}_feature_flag().
Closes: https://github.com/libfuse/libfuse/issues/1171
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
|
|
se->buf_reallocable is true when reading /dev/fuse is handled
from internal functions - we can set the variable in
fuse_session_receive_buf_internal(). With that we also
don't need to have it an _Atomic variable anymore.
In _fuse_session_receive_buf() we can use "bool internal"
to check if the buffer can be re-allocated.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
This fixes dynamic buffer allocation in commit
0e0f43b79b9b ("Reallocate fuse_session buffer...")
I noticed that when I increased the default fuse buf size as
possible in recent kernels.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
libfuse-3.17 introduced several functions that should only be called via
inlined helper functions, never directly. To enforce this, these functions
were declared within the inlined functions. However, this triggers the
compiler warning "-Werror=nested-externs".
While this warning is valid, the nested declarations were intentional to
prevent direct usage of these functions. Rather than suppressing the
warning with pragmas, move these function declarations outside the helper
functions while maintaining the intended access restrictions through other
means.
Closes: https://github.com/libfuse/libfuse/issues/1134
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
|
|
Similar previous renames to fuse_main_real_versioned,
but here for the low level fuse_session_new.
Also remove symbol versioned as part of "fuse_session_new"
as that function is not part of the official API/ABI
and to allow easier access with dlopen/dlsym.
Also switch back to a macro fuse_session_new, just in
case some code has some expectations on that.
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
|
|
The comment is probably a leftover from older versions, as the application
of the 32 bits of conn.capable_ext to conn.capable is done before function
convert_to_conn_want_ext() is invoked.
Signed-off-by: Luis Henriques <luis@igalia.com>
|
|
Because conn.want_ext is a uint64_t, copying it into a uint32_t may result
in truncating it's value. This patch fixes a bug in do_init() where the
32 bits copy is again converted into a 64 bits value, because it will be
used in convert_to_conn_want_ext().
Signed-off-by: Luis Henriques <luis@igalia.com>
|
|
_fuse_new() is not supposed to be called by external users
outside of internal functions or static inlined functions.
This also removes several functions from lib/fuse_versionscript
which where added and exported by
commit 58f85bfa9b7d ("Add in the libfuse version a program...)
as these are libfuse internal only.
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
|
|
The previous fields are left for ABI compatibility, although
it is not beautiful to add that complexity when we have to increase
the so-version as we had ABI breakage anyway.
example/printcap is simplified to use an array, as every line would
have needed to be modified anyway. Missing 'FUSE_CAP_PASSTHROUGH'
was added.
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
|
|
We don't want FUSE_MAXOP calculated at compilation time,
as the ABI limit is CUSE_INIT - better use that value
directly.
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
|
|
_int can be confused with 'integer'
|
|
Read/writes IOs should be page aligned as fuse server
might need to copy data to another buffer otherwise in
order to fulfill network or device storage requirements.
Simple reproducer is example/passthrough*
and opening a file with O_DIRECT - without this change
writing to that file failed with -EINVAL if the underlying
file system was using ext4 (for passthrough_hp the
'passthrough' feature has to be disabled).
The mis-alignment from fuse kernel is not ideal, but we can handle
it by allocation one page more than needed and then using a buffer
that is set up to compensate for kernel misalignment.
This also only set se->buf_reallocable to true when called
by a libfuse internal caller - we do not know what
external callers are doing with the buffer - update to
commit 0e0f43b79b9b
|
|
Note that name hashes and using paths as parameters
makes it very hard to support
anonymous files in the high level API.
Known Issues:
- tests have to bail out when O_TMPFILE is not supported.
This will always be the case with high level passthrough implementations.
- test_create_and_link_tmpfile has to be skipped
due to unidentified problems with github runner
|
|
When allocating as an array of char only alignment of 1 is guaranteed but
the structure needs an alignment of 8. Specify alignment explicitly
|
|
A previous PR supported extended max writes (eg write requests larger than 1 MB)
by initializing the fuse session buffer size to use the max_pages_limit set in
/proc/sys/fs/fuse. However, this is a huge problem for machines where multiple
fuse servers may be running but only one server needs large writes. In this case,
a lot of memory will be wasted and will lead to OOM issues.
This PR does a reallocation of the session buffer transparently if the server set
"se->conn.max_write" to a value larger than 1 MiB. This is only for buffers that
are "owned" by libfuse - if the server wishes to provide its own allocated buffer
for receiving/processing requests, then it should ensure that buffer is allocated
to the proper size from the start.
Local testing showed:
echo 65535 | sudo tee /proc/sys/fs/fuse/max_pages_limit
dd if=/dev/urandom of=hello_file bs=6M count=2
write requests:
write request size is 5242880
write request size is 1048576
write request size is 5242880
write request size is 1048576
|
|
This should stop some more xfstest test failures.
|
|
Simplify setting feature flags a bit by adding a helper
function.
Also move the check for valid flags into a funtion we can re-use
in a later patch.
|
|
Add a wrapper around strtol for more rigorous error checking
and convert uses of atoi and strtol to use this instead.
|
|
max_write can be limited by se->op.init() and by the buffer size,
we use the minimum of these two.
Required se->bufsize is then set according to the determined
max_write. The current thread will have the old buffer size,
though, as it already had to the allocation to handle the
FUSE_INIT call (unless splice is used and ths variable
and related buffer is not used at all).
The given bufsize is just a hint for minimum size, allocation
could be actually larger (for example to get huge pages).
|
|
The further these lines are separated from each other the harder
it is to read the code.
There shouldn't be any code change behavior here.
|
|
A recent upstream patch [1] changed FUSE_MAX_MAX_PAGES to
FUSE_DEFAULT_MAX_PAGES_LIMIT.
Update libfuse to use FUSE_DEFAULT_MAX_PAGES_LIMIT as well
instead of FUSE_MAX_MAX_PAGES.
[1] https://lore.kernel.org/linux-fsdevel/20240923171311.1561917-1-joannelkoong@gmail.com/T/#t
|
|
Currently in libfuse, the buffer size for a fuse session is
capped at 1 MiB on a 4k page system. A recent patch
upstream [1] was merged that allows the max number of pages
per fuse request to be dynamically configurable through the
/proc/sys interface (/proc/sys/fs/fuse/max_pages_limit).
This commit adds support for this on the libfuse side to set
the fuse session buffer to take into account the max pages
limit set in /proc/sys/fs/fuse/max_pages_limit. If this
sysctl does not exist (eg older kernels), it will default to
old behavior (using FUSE_MAX_MAX_PAGES (256) as the max pages
limit). This allows for things like bigger write buffers per
request.
[1] https://lore.kernel.org/linux-fsdevel/20240923171311.1561917-1-joannelkoong@gmail.com/T/#t
|
|
The pointer did not have any sanity check.
Addresses https://github.com/libfuse/libfuse/issues/979
|
|
After reading the file /proc/$PID/task/$PID/status the buffer wasn't
terminated with a null character. This could theoretically lead to buffer
overrun by the subsequent strstr() call.
Since the contents of the proc file are guaranteed to contain the pattern
that strstr is looking for, this doesn't happen in normal situations.
Add null termination for robustness.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
ref_cnt should make the intention of this variable more clear.
|
|
The function fuse_session_process_buf_int() would do much things
for FUSE_INTERRUPT requests, even there are no FUSE_INTERRUPT requests:
1. check every non-FUSE_INTERRUPT request and add these requests to the
linked list(se->list) under a big lock(se->lock).
2. the function fuse_free_req() frees every request and remove them from
the linked list(se->list) under a bing lock(se->lock).
These operations are not meaningful when there are no FUSE_INTERRUPT requests,
and have a great impact on the performance of fuse filesystem because the big
lock for each request.
In some cases, FUSE_INTERRUPT requests are infrequent, even none at all.
Besides, the user-defined filesystem may do nothing for FUSE_INTERRUPT requests.
And the kernel side has the option "no_interrupt" in struct fuse_conn. This kernel option
can be enabled by return ENOSYS in libfuse for the reply of FUSE_INTERRUPT request.
But I don't find the code to enable the "no_interrupt" kernel option in libfuse.
So add the no_interrupt support, and when this operaion is enabled:
1. remove the useless locking operaions and list operations.
2. return ENOSYS for the reply of FUSE_INTERRUPT request to inform the kernel to disable
FUSE_INTERRUPT request.
|
|
Fixes: 73cd124d0408 ("Add clone_fd to custom IO (#927)")
Signed-off-by: Xiaoguang Wang <lege.wang@jaguarmicro.com>
|
|
Add support for filesystem passthrough read/write of files.
When the FUSE_PASSTHROUGH capability is enabled, the FUSE server may
decide, while handling the "open" or "create" requests, if the given
file can be accessed by that process in "passthrough" mode, meaning that
all the further read and write operations would be forwarded by the
kernel directly to the backing file rather than to the FUSE server.
All requests other than read or write are still handled by the server.
This allows for an improved performance on reads and writes, especially
in the case of reads at random offsets, for which no (readahead)
caching mechanism would help, reducing the performance gap between FUSE
and native filesystem access.
Extend also the passthrough_hp example with the new passthrough feature.
This example opens a kernel backing file per FUSE inode on the first
FUSE file open of that inode and closes the backing file on the release
of the last FUSE file on that inode.
All opens of the same inode passthrough to the same backing file.
A combination of fi->direct_io and fi->passthrough is allowed.
It means that read/write operations go directly to the server, but mmap
is done on the backing file.
This allows to open some fds of the inode in passthrough mode and some
fd of the same inode in direct_io/passthrough_mmap mode.
Signed-off-by: Alessio Balsini <balsini@android.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
|
|
The API stays the same, the libfuse version comes from
inlined functions, which are defined fuse_lowlevel.h
and fuse.h. As these inlined functions are defined in the header
files they get added into the application, similar as if these
were preprocessor macros.
Macro vs inlined function is then just a style issue - I personally
prefer the latter.
fuse_session_new() -> static inlinei, in the application
_fuse_session_new -> inside of libfuse
fuse_new() -> static inline, in the application
_fuse_new() -> inside of libfuse
Note: Entirely untested is the fuse 30 api - we need a test
for it. And we do not have any ABI tests at all.
Signed-off-by: Bernd Schubert <bernd.schubert@fastmail.fm>
|
|
If the file system doesn't provide a ->open or an ->opendir, and the
kernel supports FUSE_CAP_NO_OPEN_SUPPORT or FUSE_CAP_NO_OPENDIR_SUPPORT,
allow the implementation to set FUSE_CAP_NO_OPEN*_SUPPORT on conn->want
in order to automatically get this behavior. Expand the documentation
to be more explicit about the behavior of libfuse in the different cases
WRT this capability.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
|
|
If user updates conn->max_write in fuse_lowlevel_ops' init() method, do_init()
will miss the "conn.max_write > bufsize - FUSE_BUFFER_HEADER_SIZE" judgment,
and ->init method will be called after it, which obviously is a bug.
Signed-off-by: Xiaoguang Wang <lege.wang@jaguarmicro.com>
Co-authored-by: Xiaoguang Wang <lege.wang@jaguarmicro.com>
|
|
This just adds in the basic handler, but does not
use it yet in examples.
|
|
User may still need to mount same fuse filesystem after umounting
it(In this case, the userspace filesystem server needs to keep live),
and after handling FUSE_DESTROY message, new FUSE_INIT message may come,
so need to reset got_init to be zero.
Signed-off-by: Xiaoguang Wang <lege.wang@jaguarmicro.com>
|
