- 30 Mar, 2021 1 commit
-
-
Brian Behlendorf authored
Increase the version to 2.1.99 to indicate the master branch is newer than the 2.1.x release. This ensures packages built from master branch are considered to be newer than the last release. Signed-off-by:Brian Behlendorf <behlendorf1@llnl.gov>
-
- 29 Mar, 2021 1 commit
-
-
Brian Behlendorf authored
New features: - Distributed Spare (dRAID) Feature - Added "compatibility" property for zpool feature sets - Added zpool_influxdb command to collect zpool statistics Signed-off-by:
-
- 26 Mar, 2021 5 commits
-
-
наб authored
When a child process is killed waitpid() must be called on the pid the reap the zombie process. Update BUGS section to reflect reality by replacing "zedlets aren't time limited with "zedlets can be interrupted". Reviewed-by:
Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #11769 Closes #11798
-
Matthew Ahrens authored
For gang blocks, `DVA_GET_ASIZE()` is the total space allocated for the gang DVA including its children BP's. The space allocated at each DVA's vdev/offset is `vdev_psize_to_asize(vd, SPA_GANGBLOCKSIZE)`. This commit makes this relationship more clear by using a helper function, `vdev_gang_header_asize()`, for the space allocated at the gang block's vdev/offset. Reviewed-by:
Matthew Ahrens <mahrens@delphix.com> Closes #11744
-
Matthew Ahrens authored
When specifying the name of a RAIDZ vdev on the command line, it can be specified as raidz-<vdevID> or raidzP-<vdevID>. e.g. `zpool clear poolname raidz-0` or `zpool clear poolname raidz2-0` If the parity is specified in the vdev name, it should match the actual parity of that RAIDZ vdev, otherwise the command should fail. This commit makes it so. Reviewed-by:
Stuart Maybee <stuart.maybee@comcast.net> Signed-off-by:
-
Luis Henriques authored
Other (all?) Linux filesystems seem to return -EPERM instead of -EACCESS when trying to set FS_APPEND_FL or FS_IMMUTABLE_FL without the CAP_LINUX_IMMUTABLE capability. This was detected by generic/545 test in the fstest suite. Reviewed-by:
Luis Henriques <henrix@camandro.org> Closes #11791
-
Jessica Clarke authored
Arm-based Macs are like FreeBSD and provide a full 64-bit stat from the start, so have no stat64 variants. Thus, define stat64 and fstat64 as aliases for the normal versions. Reviewed-by:
Ryan Moeller <ryan@iXsystems.com> Signed-off-by:
Jessica Clarke <jrtc27@jrtc27.com> Closes #11771
-
- 22 Mar, 2021 2 commits
-
-
Andrea Gelmini authored
Reviewed-by:
Andrea Gelmini <andrea.gelmini@gelma.net> Closes #11775
-
Andrea Gelmini authored
Reviewed-by:
-
- 20 Mar, 2021 10 commits
-
-
Alexander Motin authored
To make better predictions on parallel workloads dmu_zfetch() should be called as early as possible to reduce possible request reordering. In particular, it should be called before dmu_buf_hold_array_by_dnode() calls dbuf_hold(), which may sleep waiting for indirect blocks, waking up multiple threads same time on completion, that can significantly reorder the requests, making the stream look like random. But we should not issue prefetch requests before the on-demand ones, since they may get to the disks first despite the I/O scheduler, increasing on-demand request latency. This patch splits dmu_zfetch() into two functions: dmu_zfetch_prepare() and dmu_zfetch_run(). The first can be executed as early as needed. It only updates statistics and makes predictions without issuing any I/Os. The I/O issuance is handled by dmu_zfetch_run(), which can be called later when all on-demand I/Os are already issued. It even tracks the activity of other concurrent threads, issuing the prefetch only when _all_ on-demand requests are issued. For many years it was a big problem for storage servers, handling deeper request queues from their clients, having to either serialize consequential reads to make ZFS prefetcher usable, or execute the incoming requests as-is and get almost no prefetch from ZFS, relying only on deep enough prefetch by the clients. Benefits of those ways varied, but neither was perfect. With this patch deeper queue sequential read benchmarks with CrystalDiskMark from Windows via iSCSI to FreeBSD target show me much better throughput with almost 100% prefetcher hit rate, comparing to almost zero before. While there, I also removed per-stream zs_lock as useless, completely covered by parent zf_lock. Also I reused zs_blocks refcount to track zf_stream linkage of the stream, since I believe previous zs_fetch == NULL check in dmu_zfetch_stream_done() was racy. Delete prefetch streams when they reach ends of files. It saves up to 1KB of RAM per file, plus reduces searches through the stream list. Block data prefetch (speculation and indirect block prefetch is still done since they are cheaper) if all dbufs of the stream are already in DMU cache. First cache miss immediately fires all the prefetch that would be done for the stream by that time. It saves some CPU time if same files within DMU cache capacity are read over and over. Reviewed-by:
Adam Moss <c@yotes.com> Reviewed-by:
Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Closes #11652
-
Chunwei Chen authored
If TX_WRITE is create on a file, and the file is later deleted and a new directory is created on the same object id, it is possible that when zil_commit happens, zfs_get_data will be called on the new directory. This may result in panic as it tries to do range lock. This patch fixes this issue by record the generation number during zfs_log_write, so zfs_get_data can check if the object is valid. Reviewed-by:
Chunwei Chen <david.chen@nutanix.com> Closes #10593 Closes #11682
-
Andrew authored
Commit 235a8565 introduced a regression in evaluation of POSIX modes that require group DENY entries in the internal ZFS ACL. An example of such a POSX mode is 007. When write_implies_delete_child is set, then ACE_WRITE_DATA is added to `wanted_dirperms` in prior to calling zfs_zaccess_common(). This occurs is zfs_zaccess_delete(). Unfortunately, when zfs_zaccess_aces_check hits this particular DENY ACE, zfs_groupmember() is checked to determine whether access should be denied, and since zfs_groupmember() always returns B_TRUE on Linux and so this check is failed, resulting ultimately in EPERM being returned. Reviewed-by:
Andrew Walker <awalker@ixsystems.com> Closes #11760
-
Palash Gandhi authored
This change adds a new test that covers a bug fix in the binary search in the redacted send resume logic that causes a kernel panic. The bug was fixed in https://github.com/openzfs/zfs/pull/11297 . Reviewed-by:
John Kennedy <john.kennedy@delphix.com> Signed-off-by:
Palash Gandhi <palash.gandhi@delphix.com> Closes #11764
-
Martin Matuška authored
The FreeBSD boot loader relies on the bootfs property and is capable of booting from removed (indirect) vdevs. Reviewed-by Eric van Gyzen Reviewed-by:
Martin Matuska <mm@FreeBSD.org> Closes #11763
-
Ryan Moeller authored
Reviewed-by:
George Melikov <mail@gmelikov.ru> Signed-off-by:
-
Brian Atkinson authored
It used to be required to pass a enum km_type to kmap_atomic() and kunmap_atomic(), however this is no longer necessary and the wrappers zfs_k(un)map_atomic removed these. This is confusing in the ABD code as the struct abd_iter member iter_km no longer exists and the wrapper macros simply compile them out. Reviewed-by:
Brian Atkinson <batkinson@lanl.gov> Closes #11768
-
Serapheim Dimitropoulos authored
= Motivation We've noticed several zloop crashes within Delphix generated due to the following sequence of events: - A device gets expanded and new metaslabas are allocated for it. These metaslabs go through `metaslab_init()` but haven't gone through `metaslab_sync_done()` yet. This meas that the only range tree that's actually set is the `ms_allocatable`. All the others are NULL. - A vdev_initialization is issues and `vdev_initialize_thread` starts processing one of these new metaslabs of the expanded vdev. - As part of `vdev_initialize_calculate_progress()` we call into `metaslab_load()` and `metaslab_load_impl()` which in turn tries to dereference the metaslabs trees that are still NULL and therefore we crash. The same failure can come up from the `vdev_trim` code paths. = This Patch We considered the following solutions to deal with this issue: [A] Add logic to `vdev_initialize/trim` to skip those new metaslabs. We decided against this as it would be good to avoid exposing this lower-level detail to higer-level operations. [B] Have `metaslab_load_impl()` return early for new metaslabs and thus never touch those range_trees that are NULL at that time. This seemed more of a work-around for the bug and not a clear-cut solution. [C] Refactor our logic so all metaslabs have their range_trees created at the time of their creatin in `metaslab_init()`. In this patch we decided to go with [C] because: (1) It doesn't expose more metaslab details to higher level operations such as vdev initialize and trim. (2) The current behavior of creating the range trees lazily in `metaslab_sync_done()` is unnecessarily complicated. (3) Always initializing the metaslab range_trees makes other parts of the codebase cleaner. For example, we used to use `ms_freed` as the reference value for knowing whether all the range_trees have been initialized. Now we no longer need to do that check in most places (and in the few that we do we use the `ms_new` boolean field now which is more readable). = Side Changes Probably due to a mismerge we set `ms_loaded` to `B_TRUE` twice in `metasloab_load_impl()`. In this patch we remove the extraneous assignment. Reviewed-by:Serapheim Dimitropoulos <serapheim@delphix.com> Closes #11737
-
Coleman Kane authored
The BIO_MAX_PAGES macro is being retired in favor of a bio_max_segs() function that implements the typical MIN(x,y) logic used throughout the kernel for bounding the allocation, and also the new implementation is intended to be signed-safe (which the former was not). Reviewed-by:
Tony Hutter <hutter2@llnl.gov> Reviewed-by:
Coleman Kane <ckane@colemankane.org> Closes #11765
-
Coleman Kane authored
In Linux 5.12, the filesystem API was modified to support ipmapped mounts by adding a "struct user_namespace *" parameter to a number functions and VFS handlers. This change adds the needed autoconf macros to detect the new interfaces and updates the code appropriately. This change does not add support for idmapped mounts, instead it preserves the existing behavior by passing the initial user namespace where needed. A subsequent commit will be required to add support for idmapped mounted. Reviewed-by:
-
- 19 Mar, 2021 1 commit
-
-
Matthew Ahrens authored
The RAIDZ and DRAID code is responsible for reporting checksum errors on their child vdevs. Checksum errors represent events where a disk returned data or parity that should have been correct, but was not. In other words, these are instances of silent data corruption. The checksum errors show up in the vdev stats (and thus `zpool status`'s CKSUM column), and in the event log (`zpool events`). Note, this is in contrast with the more common "noisy" errors where a disk goes offline, in which case ZFS knows that the disk is bad and doesn't try to read it, or the device returns an error on the requested read or write operation. RAIDZ/DRAID generate checksum errors via three code paths: 1. When RAIDZ/DRAID reconstructs a damaged block, checksum errors are reported on any children whose data was not used during the reconstruction. This is handled in `raidz_reconstruct()`. This is the most common type of RAIDZ/DRAID checksum error. 2. When RAIDZ/DRAID is not able to reconstruct a damaged block, that means that the data has been lost. The zio fails and an error is returned to the consumer (e.g. the read(2) system call). This would happen if, for example, three different disks in a RAIDZ2 group are silently damaged. Since the damage is silent, it isn't possible to know which three disks are damaged, so a checksum error is reported against every child that returned data or parity for this read. (For DRAID, typically only one "group" of children is involved in each io.) This case is handled in `vdev_raidz_cksum_finish()`. This is the next most common type of RAIDZ/DRAID checksum error. 3. If RAIDZ/DRAID is not able to reconstruct a damaged block (like in case 2), but there happens to be additional copies of this block due to "ditto blocks" (i.e. multiple DVA's in this blkptr_t), and one of those copies is good, then RAIDZ/DRAID compares each sector of the data or parity that it retrieved with the good data from the other DVA, and if they differ then it reports a checksum error on this child. This differs from case 2 in that the checksum error is reported on only the subset of children that actually have bad data or parity. This case happens very rarely, since normally only metadata has ditto blocks. If the silent damage is extensive, there will be many instances of case 2, and the pool will likely be unrecoverable. The code for handling case 3 is considerably more complicated than the other cases, for two reasons: 1. It needs to run after the main raidz read logic has completed. The data RAIDZ read needs to be preserved until after the alternate DVA has been read, which necessitates refcounts and callbacks managed by the non-raidz-specific zio layer. 2. It's nontrivial to map the sections of data read by RAIDZ to the correct data. For example, the correct data does not include the parity information, so the parity must be recalculated based on the correct data, and then compared to the parity that was read from the RAIDZ children. Due to the complexity of case 3, the rareness of hitting it, and the minimal benefit it provides above case 2, this commit removes the code for case 3. These types of errors will now be handled the same as case 2, i.e. the checksum error will be reported against all children that returned data or parity. Reviewed-by:
-
- 18 Mar, 2021 3 commits
-
-
Mateusz Guzik authored
Avoids tripping on asserts when doing pool recovery. Reviewed-by:
Mateusz Guzik <mjguzik@gmail.com> Closes #11739
-
Matthew Ahrens authored
The `rr_code` field in `raidz_row_t` is unused. This commit removes the field, as well as the code that's used to set it. Reviewed-by:
-
Ryan Moeller authored
Don't handle (incorrectly) kmem_zalloc() failure. With KM_SLEEP, will never return NULL. Free the data allocated for non-virtual kstats when deleting the object. Reviewed-by:
-
- 16 Mar, 2021 5 commits
-
-
Adam D. Moss authored
zhold() wraps igrab() on Linux, and igrab() may fail when the inode is in the process of being deleted. This means zhold() must only be called when a reference exists and therefore it cannot be deleted. This is the case for all existing consumers so add a VERIFY and a comment explaining this requirement. Reviewed-by:
-
Dries Michiels authored
Update supported FreeBSD versions in documentation. Reviewed-by:
Dries Michiels <driesm.michiels@gmail.com> Closes #11718
-
gldisater authored
The man page was missing these two permissions. Add the missing permissions to the man page. Reviewed-by:
Jeremy Faulkner <gldisater@gldis.ca> Closes #11727
-
Ryan Moeller authored
Create a new section of tests to run with acltype=off. For now the only test we have is for the DOS mode READONLY attribute on FreeBSD. Reviewed-by:
-
Don Brady authored
To make use of zfs_refcount_held tunable it should be a module parameter in open-zfs. Also, since the macros will auto-generate OS specific tunables, removed the existing zfs_refcount_held reference in module/os/freebsd/zfs/sysctl_os.c. Reviewed-by:
Allan Jude <allan@klarasystems.com> Signed-off-by:
Don Brady <don.brady@delphix.com> Closes #11753
-
- 13 Mar, 2021 4 commits
-
-
Ryan Moeller authored
You can't use user_run to eval ksh functions defined in libtest unless you include libtest in the user shell. Fix xattr_003_neg by: * include libtest in the user shell * *then* run get_xattr * assert this fails * use variables for filenames so they don't change in the user's shell * don't log the contents of /etc/passwd * cleanup all byproducts Reviewed-by:
-
Ryan Moeller authored
The current user_run often does not work as expected. Commands are run in a different shell, with a different environment, and all output is discarded. Simplify user_run to retain the current environment, eliminate eval, and feed the command string into ksh. Enhance the logging for user_run so we can see out and err. Reviewed-by:
-
Mariusz Zaborski authored
Add parsing of the rewind options. When I was upstreaming the change [1], I omitted the part where we detect that the pool should be rewind. When the FreeBSD repo has synced with the OpenZFS, this part of the code was removed. [1] FreeBSD repo: 277f38abffc6a8160b5044128b5b2c620fbb970c [2] OpenZFS repo: f2c027bd External-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254152 Originally reviewed by: tsoome, allanjude Originally reviewed by: kevans (ok from high-level overview) Reviewed-by:
Mariusz Zaborski <oshogbo@vexillium.org> Closes #11730
-
Ryan Moeller authored
Resolve some oddities in zfsdev_close() which could result in a panic and were not present in the equivalent function for Linux. - Remove unused definition ZFS_MIN_MINOR - FreeBSD: Simplify zfsdev state destruction - Assert zs_minor is valid in zfsdev_close - Make locking around zfsdev state match Linux Reviewed-by:
-
- 12 Mar, 2021 8 commits
-
-
Mateusz Guzik authored
This deserializes otherwise non-contending operations. The previous scheme of using 17 locks hashed by curthread runs into conflicts very quickly. Check the pull request for sample results. Reviewed-by:
Matt Macy <mmacy@FreeBSD.org> Signed-off-by:
-
Mateusz Guzik authored
This will allow platforms to implement it as they see fit, in particular in a different manner than rrm locks. Reviewed-by:
-
Mateusz Guzik authored
Reviewed-by:
-
Mateusz Guzik authored
They are not very useful and hard to implement in the rms routine the code is about to start using. Reviewed-by:
-
Mateusz Guzik authored
1. even up ifdefs 2. drop the arguably useless teardown lock asserts -- nothing else checks for it Reviewed-by:
-
Mateusz Guzik authored
They are expected to fail only in corner cases. Reviewed-by:
-
George Wilson authored
Importing a pool using the cachefile is ideal to reduce the time required to import a pool. However, if the devices associated with a pool in the cachefile have changed, then the import would fail. This can easily be corrected by doing a normal import which would then read the pool configuration from the labels. The goal of this change is make importing using a cachefile more resilient and auto-correcting. This is accomplished by having the cachefile import logic automatically fallback to reading the labels of the devices similar to a normal import. The main difference between the fallback logic and a normal import is that the cachefile import logic will only look at the device directories that were originally used when the cachefile was populated. Additionally, the fallback logic will always import by guid to ensure that only the pools in the cachefile would be imported. External-issue: DLPX-71980 Reviewed-by:
George Wilson <gwilson@delphix.com> Closes #11716
-
Martin Matuška authored
The manual page change in ecc277cf has introduced whitespace on line ends. Reviewed-by:
-
