- 30 Apr, 2020 2 commits
-
-
Tony Hutter authored
META file and changelog updated. Signed-off-by:Tony Hutter <hutter2@llnl.gov> TEST_ZIMPORT_SKIP="yes"
-
George Amanakis authored
Otherwise when running with reference_tracking_enable=TRUE mounting and unmounting an encrypted dataset panics with: Call Trace: dump_stack+0x66/0x90 slab_err+0xcd/0xf2 ? __kmalloc+0x174/0x260 ? __kmem_cache_shutdown+0x158/0x240 __kmem_cache_shutdown.cold+0x1d/0x115 shutdown_cache+0x11/0x140 kmem_cache_destroy+0x210/0x230 spl_kmem_cache_destroy+0x122/0x3e0 [spl] zfs_refcount_fini+0x11/0x20 [zfs] spa_fini+0x4b/0x120 [zfs] zfs_kmod_fini+0x6b/0xa0 [zfs] _fini+0xa/0x68c [zfs] __x64_sys_delete_module+0x19c/0x2b0 do_syscall_64+0x5b/0x1a0 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Reviewed-By:
Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-By:
Tom Caputi <tcaputi@datto.com> Signed-off-by:
George Amanakis <gamanakis@gmail.com> Closes #10246
-
- 28 Apr, 2020 24 commits
-
-
Brian Behlendorf authored
Commit https://github.com/torvalds/linux/commit/3d745ea5 simplified the blk_alloc_queue() interface by updating it to take the request queue as an argument. Add a wrapper function which accepts the new arguments and internally uses the available interfaces. Other minor changes include increasing the Linux-Maximum to 5.6 now that 5.6 has been released. It was not bumped to 5.7 because this release has not yet been finalized and is still subject to change. Added local 'struct zvol_state_os *zso' variable to zvol_alloc. Reviewed-by:
George Melikov <mail@gmelikov.ru> Reviewed-by:
-
Matthew Macy authored
A struct rangelock already exists on FreeBSD. Add a zfs_ prefix as per our convention to prevent any conflict with existing symbols. This change is a follow up to 2cc479d0 . Reviewed-by:
Matt Ahrens <matt@delphix.com> Reviewed-by:
Matt Macy <mmacy@FreeBSD.org> Closes #9534
-
Arvind Sankar authored
When zfs is built in-tree using --enable-linux-builtin, the compile commands are executed from the kernel build directory. If the build directory is different from the kernel source directory, passing -Ifs/zfs/icp will not find the headers as they are not present in the build directory. Fix this by adding @abs_top_srcdir@ to pull the headers from the zfs source tree instead. Reviewed-by:
Arvind Sankar <nivedita@alum.mit.edu> Closes #10021
-
Attila Fülöp authored
There are a couple of x86_64 architectures which support all needed features to make the accelerated GCM implementation work but the MOVBE instruction. Those are mainly Intel Sandy- and Ivy-Bridge and AMD Bulldozer, Piledriver, and Steamroller. By using MOVBE only if available and replacing it with a MOV followed by a BSWAP if not, those architectures now benefit from the new GCM routines and performance is considerably better compared to the original implementation. Reviewed-by:
Adam D. Moss <c@yotes.com> Signed-off-by:
Attila Fülöp <attila@fueloep.org> Followup #9749 Closes #10029
-
Attila Fülöp authored
Currently SIMD accelerated AES-GCM performance is limited by two factors: a. The need to disable preemption and interrupts and save the FPU state before using it and to do the reverse when done. Due to the way the code is organized (see (b) below) we have to pay this price twice for each 16 byte GCM block processed. b. Most processing is done in C, operating on single GCM blocks. The use of SIMD instructions is limited to the AES encryption of the counter block (AES-NI) and the Galois multiplication (PCLMULQDQ). This leads to the FPU not being fully utilized for crypto operations. To solve (a) we do crypto processing in larger chunks while owning the FPU. An `icp_gcm_avx_chunk_size` module parameter was introduced to make this chunk size tweakable. It defaults to 32 KiB. This step alone roughly doubles performance. (b) is tackled by porting and using the highly optimized openssl AES-GCM assembler routines, which do all the processing (CTR, AES, GMULT) in a single routine. Both steps together result in up to 32x reduction of the time spend in the en/decryption routines, leading up to approximately 12x throughput increase for large (128 KiB) blocks. Lastly, this commit changes the default encryption algorithm from AES-CCM to AES-GCM when setting the `encryption=on` property. Reviewed-By:
Jason King <jason.king@joyent.com> Reviewed-By:
Richard Laager <rlaager@wiktel.com> Signed-off-by:
-
Fabio Scaccabarozzi authored
In zfs_write(), the loop continues to the next iteration without accounting for partial copies occurring in uiomove_iov when copy_from_user/__copy_from_user_inatomic return a non-zero status. This results in "zfs: accessing past end of object..." in the kernel log, and the write failing. Account for partial copies and update uio struct before returning EFAULT, leave a comment explaining the reason why this is done. Reviewed-by:
ilbsmart <wgqimut@gmail.com> Signed-off-by:
Fabio Scaccabarozzi <fsvm88@gmail.com> Closes #8673 Closes #10148
-
Mark Roper authored
Using zfs with Lustre, an arc_read can trigger kernel memory allocation that in turn leads to a memory reclaim callback and a deadlock within a single zfs process. This change uses spl_fstrans_mark and spl_trans_unmark to prevent the reclaim attempt and the deadlock (https://zfsonlinux.topicbox.com/groups/zfs-devel/T4db2c705ec1804ba ). The stack trace observed is: __schedule at ffffffff81610f2e schedule at ffffffff81611558 schedule_preempt_disabled at ffffffff8161184a __mutex_lock at ffffffff816131e8 arc_buf_destroy at ffffffffa0bf37d7 [zfs] dbuf_destroy at ffffffffa0bfa6fe [zfs] dbuf_evict_one at ffffffffa0bfaa96 [zfs] dbuf_rele_and_unlock at ffffffffa0bfa561 [zfs] dbuf_rele_and_unlock at ffffffffa0bfa32b [zfs] osd_object_delete at ffffffffa0b64ecc [osd_zfs] lu_object_free at ffffffffa06d6a74 [obdclass] lu_site_purge_objects at ffffffffa06d7fc1 [obdclass] lu_cache_shrink_scan at ffffffffa06d81b8 [obdclass] shrink_slab at ffffffff811ca9d8 shrink_node at ffffffff811cfd94 do_try_to_free_pages at ffffffff811cfe63 try_to_free_pages at ffffffff811d01c4 __alloc_pages_slowpath at ffffffff811be7f2 __alloc_pages_nodemask at ffffffff811bf3ed new_slab at ffffffff81226304 ___slab_alloc at ffffffff812272ab __slab_alloc at ffffffff8122740c kmem_cache_alloc at ffffffff81227578 spl_kmem_cache_alloc at ffffffffa048a1fd [spl] arc_buf_alloc_impl at ffffffffa0befba2 [zfs] arc_read at ffffffffa0bf0924 [zfs] dbuf_read at ffffffffa0bf9083 [zfs] dmu_buf_hold_by_dnode at ffffffffa0c04869 [zfs] Reviewed-by:
Mark Roper <markroper@gmail.com> Closes #9987
-
Alexander Motin authored
Attempt to run scrub or resilver on a new pool containing only special allocations (special vdev added on creation) caused infinite loop because of dsl_scan_should_clear() limiting memory usage to 5% of pool size, which it calculated accounting only normal allocation class. Addition of special and just in case dedup classes fixes the issue. Reviewed-by:
Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Closes #10106 Closes #8694
-
Brian Behlendorf authored
The crypto_cipher_init_prov and crypto_cipher_init are declared static and should not be exported by the ICP. This resolves the following warnings observed when building with the 5.4 kernel. WARNING: "crypto_cipher_init" [.../icp] is a static EXPORT_SYMBOL WARNING: "crypto_cipher_init_prov" [.../icp] is a static EXPORT_SYMBOL Reviewed-by:
-
Brian Behlendorf authored
The proc_ops structure was introduced to replace the use of of the file_operations structure when registering proc handlers. This change creates a new kstat_proc_op_t typedef for compatibility which can be used to pass around the correct structure. This change additionally adds the 'const' keyword to all of the existing proc operations structures. Reviewed-by:
-
Brian Behlendorf authored
The timestamp_truncate() function was added, it replaces the existing timespec64_trunc() function. This change renames our wrapper function to be consistent with the upstream name and updates the compatibility code for older kernels accordingly. Reviewed-by:
-
Brian Behlendorf authored
The getrawmonotonic() and getrawmonotonic64() interfaces have been fully retired. Update gethrtime() to use the replacement interface ktime_get_raw_ts64() which was introduced in the 4.18 kernel. Reviewed-by:
-
Brian Behlendorf authored
As part of the Linux kernel's y2038 changes the time_t type has been fully retired. Callers are now required to use the time64_t type. Rather than move to the new type, I've removed the few remaining places where a time_t is used in the kernel code. They've been replaced with a uint64_t which is already how ZFS internally handled these values. Going forward we should work towards updating the remaining user space time_t consumers to the 64-bit interfaces. Reviewed-by:
-
Romain Dolbeau authored
-fno-common is the new default in GCC 10, replacing -fcommon in GCC <= 9, so static data must only be allocated once. Reviewed-by:
Romain Dolbeau <romain.dolbeau@european-processor-initiative.eu> Closes #9943
-
alex authored
Issue #10090 reported that snapshots created between midnight and 1 AM are missing a padded zero in the creation property This change fixes the bug reported in issue #10090 where snapshots created between midnight and 1 AM were missing a padded zero in the creation timestamp output. The leading zero was missing because the time format string used `%k` which formats the hour as a decimal number from 0 to 23 where single digits are preceded by blanks[0] and is fixed by changing it to `%H` which formats the hour as 00-23. The difference in output is as below ``` -Thu Mar 26 0:39 2020 +Thu Mar 26 00:39 2020 ``` Reviewed-by:
Igor Kozhukhov <igor@dilos.org> Reviewed-by:
Alex John <alex@stty.io> Closes #10090 Closes #10153
-
Matthew Ahrens authored
Dedup send can only deduplicate over the set of blocks in the send command being invoked, and it does not take advantage of the dedup table to do so. This is a very common misconception among not only users, but developers, and makes the feature seem more useful than it is. As a result, many users are using the feature but not getting any benefit from it. Dedup send requires a nontrivial expenditure of memory and CPU to operate, especially if the dataset(s) being sent is (are) not already using a dedup-strength checksum. Dedup send adds developer burden. It expands the test matrix when developing new features, causing bugs in released code, and delaying development efforts by forcing more testing to be done. As a result, we are deprecating the use of `zfs send -D` and receiving of such streams. This change adds a warning to the man page, and also prints the warning whenever dedup send or receive are used. In a future release, we plan to: 1. remove the kernel code for generating deduplicated streams 2. make `zfs send -D` generate regular, non-deduplicated streams 3. remove the kernel code for receiving deduplicated streams 4. make `zfs receive` of deduplicated streams process them in userland to "re-duplicate" them, so that they can still be received. Reviewed-by:
Paul Dagnelie <pcd@delphix.com> Reviewed-by:
Matthew Ahrens <mahrens@delphix.com> Closes #7887 Closes #10117
-
Richard Laager authored
This fixes a bug where the generated zfs-functions was being included along with original zfs-functions.in in the make dist tarball. This caused an unfortunate series of events during build/packaging that resulted in the RPM-installed /etc/zfs/zfs-functions listing the paths as: ZFS="/usr/local/sbin/zfs" ZED="/usr/local/sbin/zed" ZPOOL="/usr/local/sbin/zpool" When they should have been: ZFS="/sbin/zfs" ZED="/sbin/zed" ZPOOL="/sbin/zpool" This affects init.d (non-systemd) distros like CentOS 6. /etc/default/zfs and /etc/zfs/zfs-functions are also used by the initramfs, so they need to be built even when init.d support is not. They have been moved to the (new) etc/default and (existing) etc/zfs source directories, respectively. Fixes: #9443 Co-authored-by:
-
Richard Laager authored
These are now handled in zfs-functions, so this is all duplicative and unnecessary. Signed-off-by: -
Richard Laager authored
Previously, they were being deleted in make distclean. This brings it in line with the example: https://www.gnu.org/software/automake/manual/html_node/Scripts.html Signed-off-by:
-
Ryan Moeller authored
The double-colon looked like a typo, but it's actually an obscure feature. Rules with :: may appear multiple times and are run independently of one another in the order they appear. The use of :: for distclean-local was conventional, not accidental. Add comments to indicate the intentional use of double-colon rules. Reviewed-by:
Ryan Moeller <ryan@ixsystems.com> Closes #9210
-
Richard Laager authored
This brings it in line with the example: https://www.gnu.org/software/automake/manual/html_node/Scripts.html This way, if the substitution code is changed, they should update. Signed-off-by:
-
InsanePrawn authored
Previously the generated keyload units for encryption roots with keylocation=file://* didn't contain the code to detect if the key was already loaded and would be marked failed in such situations. Move the code to check whether the key is already loaded from keylocation=prompt handling to general key loading code. Reviewed-by:
InsanePrawn <insane.prawny@gmail.com> Closes #10103
-
InsanePrawn authored
This commit refactors the systemd mount generators and makes the following major changes: - The generator now generates units for datasets marked canmount=noauto, too. These units are NOT WantedBy local-fs.target. If there are multiple noauto datasets for a path, no noauto unit will be created. Datasets with canmount=on are prioritized. - Introduces handling of new user properties which are now included in the zfs-list.cache files: - org.openzfs.systemd:requires: List of units to require for this mount unit - org.openzfs.systemd:requires-mounts-for: List of mounts to require by this mount unit - org.openzfs.systemd:before: List of units to order after this mount unit - org.openzfs.systemd:after: List of units to order before this mount unit - org.openzfs.systemd:wanted-by: List of units to add a Wants dependency on this mount unit to - org.openzfs.systemd:required-by: List of units to add a Requires dependency on this mount unit to - org.openzfs.systemd:nofail: Toggles between a wants and a requires dependency. - org.openzfs.systemd:ignore: Do not generate a mount unit for this dataset. Consult the updated man page for detailed documentation. - Restructures and extends the zfs-mount-generator(8) man page with the above properties, information on unit ordering and a license header. Reviewed-by:Antonio Russo <antonio.e.russo@gmail.com> Reviewed-by:
-
InsanePrawn authored
Silences a warning about an intentionally unquoted variable. Fixes a warning caused by strings split across lines by slightly refactoring keyloadcmd. Reviewed-by:
-
- 22 Apr, 2020 6 commits
-
-
Brian Behlendorf authored
When configuring as builtin (--enable-linux-builtin) for kernels without loadable module support (CONFIG_MODULES=n) only the object file is created. Never a loadable kmod. Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to the ZFS_LINUX_TEST_COMPILE_ALL macro. Reviewed-by:
-
Brian Behlendorf authored
Commit https://github.com/torvalds/linux/commit/9e8d42a0f accidentally converted the static inline function blkg_tryget() to GPL-only for kernels built with CONFIG_PREEMPT_RCU=y and CONFIG_BLK_CGROUP=y. Resolve the build issue by providing our own equivalent functionality when needed which uses rcu_read_lock_sched() internally as before. Reviewed-by:
-
Richard Laager authored
The correct name for the mount unit for / is "-.mount", not ".mount". Reviewed-by:
-
Matthew Ahrens authored
When growing the size of a (VMEM or KVMEM) kmem cache, spl_cache_grow() always does taskq_dispatch(spl_cache_grow_work), and then waits for the KMC_BIT_GROWING to be cleared by the taskq thread. The taskq thread (spl_cache_grow_work()) does: 1. allocate new slab and add to list 2. wake_up_all(skc_waitq) 3. clear_bit(KMC_BIT_GROWING) Therefore, the waiting thread can wake up before GROWING has been cleared. It will see that the growing has not yet completed, and go back to sleep until it hits the 100ms timeout. This can have an extreme performance impact on workloads that alloc/free more than fits in the (statically-sized) magazines. These workloads allocate and free slabs with high frequency. The problem can be observed with `funclatency spl_cache_grow`, which on some workloads shows that 99.5% of the time it takes <64us to allocate slabs, but we spend ~70% of our time in outliers, waiting for the 100ms timeout. The fix is to do `clear_bit(KMC_BIT_GROWING)` before `wake_up_all(skc_waitq)`. A future investigation should evaluate if we still actually need to taskq_dispatch() at all, and if so on which kernel versions. Reviewed-by:
Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed-by:
George Wilson <gwilson@delphix.com> Signed-off-by:
-
Richard Laager authored
If someone is using both multipathd and ZFS, they are probably using them together. Ordering the zpool imports after multipathd is ready fixes import issues for multipath configurations. Tested-by:
Mike Pastore <mike@oobak.org> Reviewed-by:
-
lorenz authored
On some systems - openSUSE, for example - there is not yet a writeable temporary file system available, so bash bails out with an error, 'cannot create temp file for here-document: Read-only file system', on the here documents in zfs-mount-generator. The simple fix is to change these into a multi-line echo statement. Reviewed-by:
Lorenz Hüdepohl <dev@stellardeath.org> Closes #9802
-
- 22 Jan, 2020 8 commits
-
-
Tony Hutter authored
META file and changelog updated. Signed-off-by: -
Tony Hutter authored
This applies the patch from: https://github.com/zfsonlinux/zfs/issues/9476#issuecomment-543854498 ...which was originally from: 9fa8b5b5 QAT related bug fixes This allows QAT to build. Signed-off-by:
-
John Poduska authored
The resilver restart test was reported as failing about 2% of the time. Two issues were found: - The event log wasn't large enough, so resilver events were missing - One 'zpool sync' wasn't enough for resilver to start after zinject Reviewed-by:
John Kennedy <john.kennedy@delphix.com> Reviewed-by:
Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by:
John Poduska <jpoduska@datto.com> Issue #9588 Closes #9677 Closes #9703
-
jwpoduska authored
If a device is participating in an active resilver, then it will have a non-empty DTL. Operations like vdev_{open,reopen,probe}() can cause the resilver to be restarted (or deferred to be restarted later), which is unnecessary if the DTL is still covered by the current scan range. This is similar to the logic in vdev_dtl_should_excise() where the DTL can only be excised if it's max txg is in the resilvered range. Reviewed-by:John Gallagher <john.gallagher@delphix.com> Reviewed-by:
-
Brian Behlendorf authored
When qat_compress() fails to allocate the required contiguous memory it mistakenly returns success. This prevents the fallback software compression from taking over and (un)compressing the block. Resolve the issue by correctly setting the local 'status' variable on all exit paths. Furthermore, initialize it to CPA_STATUS_FAIL to ensure qat_compress() always fails safe to guard against any similar bugs in the future. Reviewed-by:
-
Tony Hutter authored
Fix the zfs_receive_raw test case for zfs-0.8.3 by including the one-liner fix from loli10k described here: https://github.com/zfsonlinux/zfs/pull/9776#issuecomment-570252679 Signed-off-by:
-
Tony Hutter authored
Fix these lint warnings on zfs-0.8.3: $ make lint [module/spl/spl-vnode.c:494]: (error) Uninitialized variable: fp [module/spl/spl-vnode.c:706]: (error) Uninitialized variable: fp [module/spl/spl-vnode.c:706]: (error) Uninitialized variable: next_fp ^CMakefile:1632: recipe for target 'cppcheck' failed make: *** [cppcheck] Interrupt Signed-off-by: -
Brian Behlendorf authored
The cleanup_devices function should remove any partitions created on the device and force the partition table to be reread. This is needed to ensure that blkid has an up to date version of what devices and partitions are used by zfs. The cleanup_devices call was removed from inuse_008_pos.ksh since it operated on partitions instead of devices and was not needed. Lastly ddidecode may be called by parted and was therefore added to the constrained path. Reviewed-by:
-
