This fixes a regression caused by a typo in a change to
directory services startup.

* Fix some test breakage from recent refactor

(cherry picked from commit f078175924efaf0be677b827edd3edbc1c3dfb91)

* Adjust backport

* Undo error

---------
Co-authored-by: Brian Meagher <brian.meagher@ixsystems.com>

(cherry picked from commit dd1f4eb13bf3e7520bc513b6309b25c709361f6b)
Co-authored-by: Brian Meagher <brian.meagher@ixsystems.com>

During QE testing it was observed that in-memory idmap backend
settings of the idmap worker process was stale after reboot.

SIGHUP was insufficient to prompt reload and so this adds
a winbind restart to ensure we always have correct configuration
loaded.

(cherry picked from commit 168f025a99d4e70115ab8b6444aff862f4d235c0)
Co-authored-by: Brian Meagher <brian.meagher@ixsystems.com>

Bind mounts do not have unique device IDs and so cannot be treated
as uniquely identifying a filesystem mount tree. This commit
switches logic for getting tree to using statx return of stx_mnt_id.

filesystem.stat output is also expanded to include `mount_id` in
return.

getmntinfo is also expanded to allow optimized retrieval of mount
information by `mount_id`.

---------
Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* replace test-400 with proper test

(cherry picked from commit b559ecb156fc28069700a198146e031dc4c35adf)

* address review

(cherry picked from commit f9f444a6b91b6a03b2bc2664f833fcfe0179c9d8)

---------
Co-authored-by: Caleb <yocalebo@gmail.com>

(cherry picked from commit ea0be4a766d882302555c005fa584657543f5b20)
Co-authored-by: Caleb <yocalebo@gmail.com>

In some cases users may have root dataset with NFSv4 acltype and an
improper aclmode setting. This commit defaults to PASSTHROUGH aclmode
in this case (which allows chmod to succeed).

(cherry picked from commit 3ee67878b896971728922b4d3e62e347d61d9954)
Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* add connect_{timeout} in mismatch_disks

(cherry picked from commit de26bce40b0291fa58024190ad0c5f93ae27708f)

* fix and improve failover_/reboot.py

(cherry picked from commit 4cf51a75ec61cba40aea94eb68b4a07f87cf801c)

* fix and improve security/update.py

(cherry picked from commit f33bb1f723e43b03e4caa9a3c722d41983ef3976)

* fix failover_/disabled_reasons.py

(cherry picked from commit b7c21cfed9469c3c3ebd1bb9128e6b131e117fbe)

* fix issues found during testing

(cherry picked from commit ae7eafea837a36b7451149c12493122f752080a4)

* update fips key properly

(cherry picked from commit e84d7312fdf5054f3e972adda81aa259df5fc422)

* remove stray file

(cherry picked from commit d9ae89cbf67be0643930779eae207d943124a905)

---------
Co-authored-by: Caleb <yocalebo@gmail.com>

If using runtest.py against a previously configured HA system, and we are
skipping test_001_configure_interface, then we may want to force the
truenas_server.ip to the VIP.

(cherry picked from commit c5a7433d7c926acddc7816aa53089f1d0df7e0cf)
Co-authored-by: Brian Meagher <brian.meagher@ixsystems.com>

(cherry picked from commit 911409d6011ca8e6686c12f0ba4aecc477df39f8)
Co-authored-by: Caleb <yocalebo@gmail.com>

* Add rel_tgt_id to iSCSI target database

(cherry picked from commit 302f1ca1b151402e39b2f556c4b82717d632fc3b)

* Update src/middlewared/middlewared/plugins/iscsi_/targets.py
Co-authored-by: themylogin <themylogin@gmail.com>
(cherry picked from commit 4e9ced20bb61d84097ff470cc680c256be6e8c22)

---------
Co-authored-by: Brian Meagher <brian.meagher@ixsystems.com>
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

(cherry picked from commit 3392a69e4f3726d13668bfbbfc83846c1ab31a5e)
Co-authored-by: themylogin <themylogin@gmail.com>

* fix FIPS on HA

* failover_/boot.py -> failover_/reboot.py

* remove fips test

NAS-128174 / 24.04.0 / Make REST functions in CI use proper IP addresses on HA (by yocalebo) (#13469)

* Make REST and websocket calls VIP aware

(cherry picked from commit 7e5f70beda3f829046d2798569f8ff01c0bc0156)
(cherry picked from commit 8dc4a6ae519f7ed21e3d840e5c5a9067e07bd549)

* Fix syntax for old python

(cherry picked from commit 2937ffde6a26e7b905e79b6910b41cea84365aa1)
(cherry picked from commit 3c765233711ab8f857a632d3ceb69459648995ca)

* Fix SSH for VIP

(cherry picked from commit 977344916b0dbf1d195bf73febcc5bba1999edcc)
(cherry picked from commit fa819960d25dfae8c815f34c703c69850afacc1d)

* use server variable instead of calling host()

(cherry picked from commit 6c8b0e828b1acf5527617045034488da670f82cb)

* flake8 while im here

(cherry picked from commit 3d2a8619dec2575b0c6fe5e0d7e48ad7179ff013)

* remove USE_VIP env var

(cherry picked from commit e24b235762f12d13805b091b78e1316d124f69f9)

* address review

(cherry picked from commit 90ce8412ce8c934d9522259ec2d9a95f4501bfd6)

---------
Co-authored-by: Andrew Walker <awalker@ixsystems.com>
Co-authored-by: Caleb <yocalebo@gmail.com>

(cherry picked from commit ed7ceb26e81d3e6685f0e97aafc8a19705e77057)
Co-authored-by: Caleb <yocalebo@gmail.com>

* add new changes for new jenkins to runtest.py

(cherry picked from commit ddb18da1)

* fix conftest.py to use proper IP address on HA

(cherry picked from commit 79dcc155)

* simplify test_001_ssh.py

(cherry picked from commit 50ee263e)

* fix and simplify test_002

(cherry picked from commit 78f12870)

* fix and simplify test_003

(cherry picked from commit cc85d0cb)

* fix and simplify test_005

(cherry picked from commit 6c6cc072)

* fix and simplify test_006

(cherry picked from commit ccd7302a)

* fix and improve API WS client

(cherry picked from commit 06840691

)

* address review

(cherry picked from commit 8f8b402c939f661f5cc8b8b62ad62c08eb481c2a)

* be less strict on finding default gateway

(cherry picked from commit 25fece64a87f8eac5001c4c3af172edfa7cb06d7)

---------
Co-authored-by: Caleb <yocalebo@gmail.com>

This commit fixes an issue where system boot id is currently changing on each middleware restart, we now introduce changes to make sure that it remains persistent across a single boot of the system itself and not just middleware.

(cherry picked from commit b143932677376940caaf578161e0765b5b487e88)
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Add basic failover service to reboot standby controller

(cherry picked from commit 872816069a8bd4d4870c56dda846a0d3c7442e66)

* Make sure we wait for standby node to come back up as and report back as ready

(cherry picked from commit ba74383b124c3f0b121c7d3b0b59c837216ab652)

* Add boot id check to ensure the standby node did actually reboot

(cherry picked from commit 23a5e9eaf0670b4a16785b9ce9e780e192cec384)

* Update job progress while rebooting standby

(cherry picked from commit 9984285bc2d7f7370a8c42317bdc542b182abc62)

* Do not allow changing FIPS settings if HA is not healthy

(cherry picked from commit 01b198dc4717a37e7273ef14d840ff9c4a5c507a)

* Add a method to retrieve boot ids

(cherry picked from commit 399fc6ea2edec8783947102410bef8a9f7a75647)

* Persistently save boot ids of nodes

(cherry picked from commit 879ee6d641a167550503a944086d9e5e1fb17712)

* Add a method which determines if reboot is required for HA

(cherry picked from commit de4657a740be4e5a34668902d4fe541405393d6e)

* If no reboot is required, remove fips_toggled key

(cherry picked from commit e5913072462bfea440c4b1bd01e9fccadeac33fa)

* Don't raise a connect error if not able to communicate with remote

(cherry picked from commit 19b7b59fb3f55a960c7177ce1e3be8c25271271c)

* Make sure we update our reboot cache keys when node becomes master:

(cherry picked from commit eda3938b92cb2bc1145947a2b092f8d35deb861e)

* Fix typo

* Add copyright header to reboot file

* Add logging statement to clarify when we are updating reboot cache

* Use become_passive to reboot standby node

* Move reboot required for failover to failover disabled reasons

* Do not add fips reboot reason to disabled reasons output endpoint

* Refactor the new service part to failover.reboot

* Add a boolean endpoint which dictates if reboot is required for HA

* Move reboot standby endpoint to failover.reboot service

* Make sure standby_reboot can only be called on master node

* Remove reason from failover.reboot.info endpoint

* Add individual reasons to disabled reasons for retrieving failover being required on ha nodes

* Fix disabled reasons endpoint logic for reboots

* Allow changing fips settings if the only reasons are rebooting because of fips

* Add cli namespace for failover.reboot service

* Simplify faiover.disabled.reasons endpoint wrt reboot logic

* Fix edge case

---------
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

(cherry picked from commit 9c195e8149de61ee0a9d473d28658a5aa2e866cf)
Co-authored-by: M. Rehan <mrehanlm93@gmail.com>

* Fix cron fields values, follow-up to NAS-125384

* Update src/middlewared/middlewared/alembic/versions/24.04/2024-04-03_14-10_cron.py
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

---------
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

(cherry picked from commit 9f84b0fa0b6d4141a152722fd837ef5764e1a46b)
Co-authored-by: M. Rehan <mrehanlm93@gmail.com>

(cherry picked from commit 773532ddc3ef0c4e0d8bbf6b14edd194a77b821b)
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

There are various parameters in which the administrator can provide
arbitrary commands and scripts to run via auxiliary parameters, some of
which may be executed as the root user. This can present a risk of
a limited administrator accidentally or intentionally performing actions
on the NAS with privileges that are higher than intended.

(cherry picked from commit 13769f03

)
Co-authored-by: Andrew Walker <awalker@ixsystems.com>

(cherry picked from commit 7058fb11f25fa210ee5df8e93317c2bc86f9414c)
Co-authored-by: themylogin <themylogin@gmail.com>

(cherry picked from commit 02dfb35da85407a9b6fa73bfa8fb23874d04b113)
Co-authored-by: themylogin <themylogin@gmail.com>

(cherry picked from commit d696fd47a9f3acf1886e32dec677cd0530e74158)
Co-authored-by: themylogin <themylogin@gmail.com>

(cherry picked from commit f09ce58f5b00a7e8ed270746f0ef202d9a3bd71f)
Co-authored-by: themylogin <themylogin@gmail.com>

standby node.
Added stop of mountd during standby mode processing.

(cherry picked from commit 6ff8f4f2b6b71e7aefc2541759d08ec78635abee)
Co-authored-by: Mark Grimes <mark.grimes@ixsystems.com>

Since the initial angelfish release in order to facilitate SMB
clustering we have stored Samba's global section (services->SMB) in the
samba registry via libsmbconf. Libsmbconf is more strict at validation
of what gets written in the configuration. Some core users have copius
smb_options with many comments in them. This shifts the function
that was used for stripping comments from auxiliary parameters from
sharing.smb to a utility function for consumption by both sharing.smb
and services.smb.

(cherry picked from commit 58a5e4eb007f2e1c319f732de882f785726c5b31)
Co-authored-by: Andrew Walker <awalker@ixsystems.com>

(cherry picked from commit 13c4f68330ea85abbc483343004d277a1bdd6aaf)
Co-authored-by: themylogin <themylogin@gmail.com>