Add script explaining that package management tools are disabled

Since our build process needs to perform some apt install commands,
delay setting this script executable until afterwards.

This commit fixes an issue where when backup node middleware restarts, we do not sync keys from master node because when we tried to sync the keys on middleware boot - we were not able to talk to active node which is highly likely because middleware has not initialised itself properly at that point, so we add a delay to ensure we only do that once middleware has booted and initialised itself properly.

(cherry picked from commit 111474c56a16e6c6f22f17c5c265cb3ad445cb5a)
Co-authored-by: themylogin <themylogin@gmail.com>

(cherry picked from commit c4c1246b22bb34427469328b48c46948c672451b)
Co-authored-by: themylogin <themylogin@gmail.com>

* Do not retrieve datasets associated with shares

* Remove sharing_task_datasets method as it's not being used anywhere now

* Add a method to cache locked datasets

* Use cached locked datasets in sharing task service and have an extra param to toggle that

* Invalidate locked datasets cache whenever any dataset is unlocked

* Rename extra parameter name to use_cached_locked_datasets

* Only cache zfs datasets if the system is ready

* Cover sharing.smb.query usages to selectively not use cache

* Make retrieving locked info attr optional by an extra parameter

* Cover sharing.smb.query usages to not retrieve locked share info where not required

* Cover sharing.nfs.query usages to not retrieve locked share info where not required and also bypassing cache where required

* Cover iscsi.extent.query usages to not retrieve locked share info where not required and also bypassing cache where required

* Cover cloudsync.query usages to not retrieve locked tasks info where not required and also bypassing cache where required

* Add locked datasets cached to pool.dataset namespace to avoid hitting process pool

* Remove usages of zfs.dataset.locked_datasets_cached

* Remove zfs.dataset.locked_datasets_cached method as it's not being used anymore

* If select is specified and lock field is not required - we should not bother to get it

* Use select where we can instead of retrieve_locked_info extra param

* Invalidate locked datasets cache whenever any dataset is locked

* Add integration test to validate sharing services lock key

* Add an integration test to verify if locked field is not specified we don't retrieve it

* Clean integration test

* Add integration test to verify cached extra parameter functioning as expected

* Fix iscsi extent volume creation parameters

This commit fixes an issue where we tried to unlock all the passphrase encrypted datasets on failover each time we imported a pool but that was bound to fail as pools which had not imported at that time would not be found and would obviously result in erroneous logging. So changes have been made to only unlock datasets of pool which has been imported at the time.

Historically, there was a configuration scenario with GELI encrypted data pools and system dataset location that would require storing the AD bind password in our configuration database. With SCALE, this configuration SMB_HA_LEGACY was formally deprecated and removed.

In early Cobia development we began to no longer keep the AD bind password in the configuration database for even transient purposes during domain join, but migration to actually drop the database column was actually omitted at that time. This commit finishes the process of removal. There are no cases where this data is used (even if somehow present in upgraded server's configuration file) and so there are no edge cases where this migration will break users.

Original PR: #12346
Jira URL: https://ixsystems.atlassian.net/browse/NAS-124687

Select logic was broken for case where the parameter we were selecting
had a null value. This fixes the behavior and adds test.

* Move default krb5cc from /tmp to /var/run/middleware
* On every boot systemd cleans /tmp directory.  This randomly results
  in deleting a newly generated system krb5cc_0

The fix is to move the 'default' location for the credential cache.
The path is middleware.utils.MIDDLEARE_RUN_DIR = /var/run/middleware

Updated the SYSTEM and USER krb5ccache
Updated krb5.conf.mako to configure kerberos to use the new path

* Add modification to krb5 stub file
Prepend 'FILE:' to ccache path

* Convert the stub from hard-coded parms to use the values from KRB_LibDefaults.
Small flake8 cleanup to runtest.py

Force RFC-1918 addresses for inter-node traffic on TrueNAS
clusters.

This commit adds changes to reduce the time to check if truecommand connection is active to 30 seconds instead of 30 minutes after setting up the interfaces and everything because the latter was way too long and system only updated the status before if truecommand.config was explicitly called. Setting it to 30 seconds works nicely and is enough to ensure the relevant wireguard interface is up and everything.

In principle in Linux this is less of a problem than in FreeBSD
because of proper inotify support, and kernel oplocks. Add a
share_type preset that provides user with dataset settings that
are likely to make SMB and NFS clients happier.

This commit removes a legacy behavior for the private method
dscache.query. In this case dscache.query calls would also retrieve the
local users datastore and return unified results. This was due to
some design constraints when replacing the user and group cache in our
legacy django webui in TrueNAS 11.2. As of SCALE the webui no longer
directly calls dscache.query to get user / group lists and so we can
safely remove this oddity.

The primary motivation for removing this legacy behavior is to fix
how query-options are applied to query results.

This commit includes two principle changes:
1) the nsupdate endpoint now allows GLOBAL addresses
2) validation for IP addresses to register happens earlier in
activedirectory.update so that we can raise proper validation error
and redirect user to either disable the automatic DNS update or fix
the server configuration prior to joining AD. As things stand, this
can cause an exception mid-join and leave server in semi-deployed
state.

Future enhancement will be to allow users to select which addresses
to register in DNS. At that point, we can safely allow global
addresses.

This commit fixes an issue where whenever a user's secret is renewed we make sure to reflect that in SSH settings as if it's not reloaded, it will continue using older secret.