NAME
manage_zone.sh - manage the configuration and lifecycle of zones and VMs
SYNOPSIS
/opt/omnicube/sbin/manage_zone.sh
-z zone
-b|-s|-r
manage_zone.sh
-z zone
-l
-h node
manage_zone.sh
-z zone
-S
-h node
-U user
-D parent_remote_dataset
manage_zone.sh
-z zone
-a|-d
-c iso
manage_zone.sh
-z zone
-a|-d
-w wwn
[-w wwn ...]
manage_zone.sh
-z zone
-a|-d
-W file
manage_zone.sh
-z zone
-L
manage_zone.sh
-z zone
[-m
ram]
[-p
vcpus]
[-n
on|off]
manage_zone.sh
[-i
name |
--instance name]
other options
DESCRIPTION
manage_zone.sh is the operator entry point for OmniCube zones, native (lipkg, lx) as well as VM brands (bhyve, kvm). It performs one operation per invocation, selected by -b, -s, -r, -l, -S, -a, -d or -L, and it can patch the RAM, VCPU and VNC attributes of a zone configuration with -m, -p and -n. -a and -d are qualified by what they act on: -c for an installation image, or -w (one or more WWNs, repeating the option) or -W (a file of them) for SCSI LUNs passed through to a VM.
The script sources /opt/omnicube/lib/common/utils.sh and inherits the shared run-level guard: at run level 0, 1, 5, 6, S or s, or when /etc/nologin exists, it logs one message and exits 0 without touching anything. Because this is an interactive tool the OC_IGNORE_RUNLEVEL=1 override matters here: an administrator doing single-user maintenance can still drive a zone by hand.
Two classes of zone are distinguished throughout, by the existence of the SMF instance svc:/service/omnicube/zone:zone:
Cluster operations use ssh. The peer list is the SMF property config/nodes of svc:/omnicube/sysadm:instance, read by the shared library into the nodes array; the local host, as returned by hostname, is always skipped. Connections use BatchMode=yes, StrictHostKeyChecking=yes and a short connect timeout, so every peer's host key must already be in /etc/ssh/ssh_known_hosts or in the invoking account's known_hosts. Remote privileged commands run under pfexec on the peer rather than by elevating the local ssh client, and every operation that depends on a peer first probes the whole ssh path by running true there, so an unusable peer aborts the command instead of stranding the zone half way.
Start (-b) and shutdown (-s)
-b sets autoboot=true for a local zone, or autoboot=false, config/adm_status = started and a refresh of the instance for a cluster managed one, then boots the zone with zoneadm boot. -s always sets autoboot=false, and for a cluster managed zone sets config/adm_status = stopped and refreshes the instance; if the zone is running, its type attribute is read and a windows guest is asked to stop first through shutdown_windows_server(8), then zoneadm shutdown runs, with zoneadm halt as a fallback when it fails.
Synchronise the configuration (-r)
Exports the zone configuration of a cluster managed zone and installs it on every other node of config/nodes; local zones are skipped, and an unreachable peer is logged and skipped so the remaining ones are still updated. On each peer the zone must be absent or in state configured, read from zoneadm list -pnic; any other state is reported and that peer skipped, so a running zone is never overwritten. The file is staged under a name created by mktemp on the peer (/tmp/manage_zone_config.XXXXXX), where pfexec zonecfg delete -F, scp and pfexec zonecfg -f then run; both staging files are removed whatever happens. The same synchronisation runs at the end of any invocation in which -m, -p or -n changed the configuration.
Relocate a SYNC zone (-l -h node)
Moves a zone whose storage the target can already reach. The target must not be the local host and the zone must be configured locally. For a cluster managed zone the three SMF properties are read first, every pool in config/zpools is locked (see ZFS lock property below), the instance is disabled with svcadm disable -s - whose stop method halts the zone, stops its VNC console relay and exports the pools - the exported configuration is pushed to the target, the local configuration and the local instance are deleted, the instance is recreated on the target with the same adm_status, type and zpools values, and svcadm enable -s is run there. For a local zone the pool named after the zone is locked, the zone is shut down (halted if it does not stop), its configuration is written to zonepath/config, the zone is detached, its configuration deleted, the VNC console relay stopped and the pool exported; the target then imports the pool, applies zonepath/config, attaches and boots the zone.
Stopping the console relay is not housekeeping: it is a precondition for the export. The relay holds zonepath/log/vnc.log open, a file inside the pool, so an export while it runs fails with "pool is busy". Both paths go through kill_zone_console (see oc_console(3)), which waits for the processes to actually exit rather than signalling and moving on.
Whether a console was published here is recorded before the teardown, and if it was, the console is re-published on the target once the zone is running there, by running activate_zone_vnc.sh(1) over ssh. A zone that had no console gets none, so a relocate neither hands an operator a console they never asked for nor silently drops one they were using. The port is not carried across because it does not need to be: it lives in the zone configuration, which travelled with the zone. TLS is deliberately not mirrored - the target's own config/vnc_tls, config/vnc_cafile and config/vnc_verify_client decide, so a node whose policy requires mutual TLS is never downgraded by a zone arriving from a laxer one. A failure to re-publish is a warning naming the command to run by hand, never a failed relocate: the zone has already moved and is running. Under -S the re-publish additionally requires that the zone was running before the move, since a zone that was stopped here is attached but not booted there.
A zone with directly attached SCSI LUNs has them checked and initialized on the target first; see Moving a zone that has LUNs attached below.
Send an ASYNC zone (-S -h node -U user -D parent)
Replicates a local zone to another node with zfs send and zfs recv, and refuses any zone that has an SMF instance. The data root is the dataset mounted at the zone's zonepath, truncated at the first path component equal to the zone name; the target is parent/leaf, the leaf being the last component of that root, that is the zone name. The root is locked, the pre-shutdown running state is recorded, and for lipkg so are the active boot environment mounted at /zones/zone/root/root and the delegated datasets named in the configuration; the zone is then shut down, the configuration blob is written to zonepath/config with its set name= dataset paths rewritten to the remote namespace, and the zone is detached, unconfigured and its console process killed. Five reported steps follow: a best-effort autosync(8) pass, whose failure is only a warning; capture of the locally set *:remotebackup, *:snapshotting, *:bckcandel, *:is_locked and *:snappolicy properties over the whole subtree, plus mountpoint and zoned for lipkg; a local zfs inherit of those properties except *:is_locked; a recursive @SEND snapshot and the transfer itself, which is a single zfs send -R piped into zfs recv -u -F when the target dataset does not exist yet, and otherwise a per-dataset walk sending each dataset incrementally from its newest snapshot marked insync=yes, or in full when it has no such base; and finally the brand-specific remote setup, where VM brands get zfs set mountpoint, zfs mount -a, zonecfg -f and zoneadm attach, while lipkg zones get a canmount=off/on bracketed replay of every source mountpoint and of zoned=on, a legacy mount of the active boot environment, then zonecfg -f and zoneadm attach. Either way the zone is booted on the target only if it was running before the send.
Afterwards the remote @SEND snapshot is destroyed, every received snapshot is marked insync=yes once per config/site prefix found among the local svc:/omnicube/sysadm instances, *:snapshotting and *:remotebackup are reapplied on the target, and both locks are released. The source datasets and their snapshots are left in place on purpose, for the operator to remove once the target is confirmed healthy.
As with -l, a zone with directly attached SCSI LUNs has them checked and initialized on the target before the zone is shut down; see Moving a zone that has LUNs attached below.
Add or remove an ISO (-a, -d with -c)
The image must exist as /iso/iso. Adding appends a read-only, nodevices lofs file system entry for that path plus a cdromN attribute; removing deletes the matching fs and attr entries. Both end with verify and commit.
Attach or detach SCSI LUNs (-a, -d with -w or -W; -L to list)
A SAN LUN can be handed to a bhyve VM directly, as a uscsi passthrough target behind an emulated virtio-scsi controller, instead of being copied into a file or a zvol. The LUN is named by its WWN and the raw device path is derived from it as /dev/rdsk/c0twwnd0p0. Both spellings of the WWN are accepted: illumos multipathing names LUNs in upper case hexadecimal while inventories and operators usually write them in lower case, so the script tries the spelling given, then upper, then lower, and records the path that actually exists on the node.
-a with -w refuses to continue when no such device node exists, which is the usual sign that the LUN is not mapped to this node or that devfsadm -Cv has not been run, and it refuses a WWN already present in the configuration under either spelling. It then writes, in one zonecfg transaction ending in verify and commit:
Initializing a LUN before it is attached
Every LUN of an attach is first put through
pfexec mpathadm modify lu -b none /dev/rdsk/c0twwnd0p0
which disables multipath load balancing on the logical unit. A LUN handed to bhyve while the mpxio layer is still balancing across its paths does not behave as the site expects, so this is a prerequisite rather than a tuning step, and nothing else performs it at attach time — the bhyve launcher does not.
On a loaded array a single call can take tens of seconds, and a batch of sixteen can therefore run for minutes with nothing else to show for itself. The step is narrated for that reason: a line naming the LUN is printed before each call, so whatever is being worked on is on screen during the wait, and a second line reports it with the elapsed time in whole seconds. An apparently hung command is almost always this, and interrupting it is not necessary — a run that does not finish leaves the zone configuration untouched.
Initialization runs after the batch has been validated and planned, so a list that was never going to fit does not first spend minutes in mpathadm, and before anything is committed, so a LUN never appears in a zone configuration until it has actually been initialized. A failure on any LUN abandons the whole attach and writes nothing; LUNs initialized before it keep -b none, which is harmless by itself and is re-applied idempotently on the next run. When /usr/sbin/mpathadm is absent the attach is refused rather than performed without initialization. Detaching does not restore load balancing.
The setting is not persistent
mpathadm modify lu sets a property of the logical unit at run time. It does not survive a reboot, and it is per node: it says nothing about the same logical unit as another node sees it. Three things therefore cover the three moments a LUN can need it.
An individual zone restart needs nothing: the node has not rebooted, so whichever of the three ran last still stands.
The lun-init service
svc:/service/omnicube/lun-init:default walks every configured zone on the node once per boot, collects the /dev/rdsk devices each one passes through — by the same rules -L uses, so all three attribute conventions are covered — and runs mpathadm modify lu -b none on each. A device shared by two zones is initialized once and reported against both. The global zone is skipped.
Its manifest makes svc:/system/zones:default and svc:/service/omnicube/zone depend on it with grouping optional_all, which is the whole design: the zone services wait for this one to reach a terminal state, so the LUNs are ready before a guest touches them, but a failure here can never keep a node's zones from booting. Starting a zone on an uninitialized LUN is bad; refusing to start any zone on the node because one LUN of one zone is unreachable is worse.
Two properties tune it.
Everything the pass does, including each device it could not initialize and the zones affected, goes to the service log and to syslog under the tag oc-lun-init. Read it with svcs -L omnicube/lun-init.
Moving a zone that has LUNs attached (-l, -S)
A zone whose disks are SAN LUNs passed straight through can only run on a node that can see those LUNs, and nothing in the move itself establishes that. Both -l and -S therefore pre-flight the LUNs on the target before any teardown starts here. The check runs after the ssh path has been proven and before the pools are locked, the instance disabled or the zone shut down, so a target that cannot take the zone costs a message and leaves the zone running where it is.
Every /dev/rdsk path the zone's configuration names is collected — from disk<n>, scsi<m>-target<j>, the raw extra<n> form, and the device resources that back them, so a zone configured under any of the conventions is covered — and each is checked on the target, in this order:
-
/usr/sbin/mpathadm must be present there. Nothing on the target will initialize the LUNs later — the launcher does not, and the lun-init service runs at system boot, which has already happened — so a target without it is a node whose guests get their LUNs while mpxio is still balancing them.
-
each LUN must have a device node. One ssh round trip probes the whole set, so half-finished SAN masking is reported in full rather than one LUN at a time. A LUN that is present under the other case spelling is reported as that, not as a missing LUN: the launcher opens the path in the configuration literally, so the fix is the configuration, not the SAN.
-
each LUN is then initialized on the target with pfexec mpathadm modify lu -b none, narrated per LUN with its elapsed time exactly as at attach time. This is a per-node setting that does not travel with the zone, so it has to be done there; doing it now moves the wait out of the zone's first boot on the target and proves that node's privilege path to the command while the move can still be called off.
Any of the three failing refuses the move with nothing changed locally. A zone with no passed-through LUNs is unaffected and says nothing.
OC_LUN_SKIP_TARGET_CHECK=1 skips the whole pre-flight, for a move an operator has reason to start anyway — masking scheduled to complete inside the window, for instance. The LUNs then have to be checked and initialized on the target by hand.
Which attr convention is written
The bhyve launcher, /opt/omnicube/lib/illumos/boot, changed how a LUN is bound to its controller in sysadm 1.12, so the form to write is not a matter of taste:
Exactly one form is written, never both. The legacy pair is not merely redundant on a 1.12+ launcher: that launcher's additional-disk loop passes diskifN straight through as a bhyve device-emulation name, so diskif3=scsi0 is emitted as -s 5:3,scsi0,path, which bhyve does not parse.
The version is read with pkg list -H omnicube/sysadm and the version field is located by shape rather than by column, because IPS inserts a publisher column when the package does not come from the preferred publisher. When it cannot be read the command aborts rather than guess.
Because -r pushes one configuration to every node, the peers listed in config/nodes are probed the same way and a cluster whose nodes straddle 1.12 is refused: no single configuration can satisfy launchers on both sides. A peer that cannot be reached is only warned about, since the synchronisation would skip it too. OC_LUN_ATTR_MODE=legacy or OC_LUN_ATTR_MODE=target forces the form and skips every probe, which is the way through a mid-upgrade cluster and the way to write a configuration deliberately for the other convention. -L honours it too, so a listing made in the same shell describes the form an attach would actually write rather than the one the version implies.
Attaching or listing also warns when the zone already carries attributes of the other convention, which is what an upgrade leaves behind: the attributes survive it untouched but the launcher stops reading them, so those disks quietly stop reaching the guest. Detaching and re-attaching each affected LUN rewrites it in the current form. Detach itself is convention-independent, so a zone configured before an upgrade stays manageable after one.
The two conventions roll over identically but NUMBER differently, and the difference is not cosmetic:
On scsi0 the two coincide for the first sixteen LUNs — disk14 with diskif14=scsi0 corresponds to scsi0-target14 — which makes them easy to conflate. They part at the first rollover.
A zone tops out at controllers times LUNs per controller, so eight times sixteen, or 128 LUNs, under either convention. A data disk already occupying a diskN slot counts against the legacy total.
An attach measures the whole batch against the remaining capacity first and reports the arithmetic:
capacity: 113 slot(s) free, 11 requested (legacy convention)
A batch larger than that is refused there, before any LUN is initialized, rather than partway through the plan — some LUNs reported as planned and a later one failing reads like a partial success, and none of them were written. -L reports the same headroom without attempting anything.
-d with -w removes every attribute naming that path, together with the diskifN partner of each diskN it removes, and the device resource. It works from the configuration rather than from the filesystem, so a LUN already unmapped on the SAN is still removable. The controller itself is left in place even when its last target goes, so that the next attach cannot reuse an index for an unrelated set of disks.
Both mark the configuration as changed and therefore trigger the synchronisation described under -r, and both warn, without failing, when the zone is running: zonecfg edits the persistent configuration, and a running bhyve process was built from the configuration as it stood at boot, so the change takes effect at the next boot of the zone.
-L prints the current map, controller by controller, showing both forms of the association so that a configuration left inconsistent by an upgrade is visible at a glance, and names the convention this node would write. It is read only, and it does not probe the peers.
It reports the remaining headroom for the convention in effect, so how many more can this zone take? is answerable without attempting an attach and reading the refusal — in the legacy convention that ceiling belongs to the zone, so a zone already carrying a dozen disks cannot take a large batch whichever controller is nominated, and reverting it to an earlier state does not necessarily help.
It also reports three inconsistencies that nothing else notices, because zonecfg type-checks none of this and the launcher does what it is told:
Only /dev/rdsk values participate, so bootdisk — a path relative to the zonepath — and any non-LUN device resource cannot produce a false report.
Attaching many LUNs at once (repeated -w, or -W)
Several LUNs can be given either by repeating -w or by naming a file with -W. The two are equivalent past the point where the list is built, and both are subject to everything described below.
-W takes a file instead of WWNs on the command line, one WWN per line. The path is checked before it is read: it must be a regular file, readable, no larger than a megabyte, and free of NUL bytes. That last test is what distinguishes text from binary here, rather than file(1)'s verdict, whose wording differs between illumos and GNU; when file(1) is installed its classification is quoted in the error message, but it is never the test. The point is that a mistyped or tab-completed path landing on a tarball, an image or a core file is rejected outright instead of producing a line-numbered complaint per line of binary. A short binary containing no NUL falls through to the per-line check, which rejects it by line number. Blank lines are ignored, and so is any line whose first non-blank character is #, so a list can be annotated and individual entries commented out. A trailing comment on a line that carries a WWN is not stripped: quietly ignoring the tail of a line that is about to become a device path is how a typo becomes the wrong LUN. A carriage return at end of line is tolerated, so a file edited on Windows works. The whole file is parsed before anything is attempted and every bad or repeated line is reported with its number, rather than only the first.
A batch is applied all or nothing. Every WWN is resolved and checked — device node present, not already attached, controller and index limits not exceeded — and only then is a single zonecfg transaction built and committed. One unmapped LUN in a list of fifty therefore leaves the configuration exactly as it was, which is what makes a failed run safe to repeat once the list is fixed. Controllers roll over within the batch, so a list of twenty LUNs against an empty zone fills scsi0 with sixteen and puts the remaining four on scsi1.
-W works with -d as well, and is subject to the same all-or-nothing rule: if any listed WWN is not attached to the zone, none of them are detached.
RAM, VCPUs and VNC (-m, -p, -n)
Each rewrites an existing zone attribute with zonecfg select attr name=...; the attribute must already be present in the configuration, as created from vm_tpl.cfg(5). A successful change triggers the synchronisation described under -r.
ZFS lock property
-l and -S take an advisory lock by setting the ZFS user property PROPPREFIX:is_locked to true on the datasets they are about to move, where PROPPREFIX is the value of the SMF property config/site. If a target already reads true, the operation reports that the pool is locked by another process and aborts without changing anything. While set, the property makes autosnap(8), autocleansnap(8) and autosync(8) skip those datasets.
Every error path releases the property with a recursive zfs inherit -r, locally while the pool is still imported here or over ssh on the target once it has been handed over, and verifies the result by value; the recursion matters because on a received dataset the property can sit deeper in the tree with SOURCE=received. When the pool is between export and import, or the peer is unreachable, the release is best effort and a warning names the datasets and the exact pfexec zfs inherit -r command to run by hand. Only a SIGKILL during a move can leave the property set.
-S additionally installs an EXIT, HUP, INT and TERM trap that destroys data_root@SEND however the script terminates, and destroys a stale @SEND before taking a new one, so an interrupted run never blocks the next. There is no lock directory under /var/run/omnicube: this property is the only concurrency control in the command.
OPTIONS
EXAMPLES
Example 1: start and stop a zone
manage_zone.sh -z zbeio01 -b manage_zone.sh -z zbeio01 -s
Example 2: resize a VM and enable its VNC console
manage_zone.sh -z zbeio01 -m 16 -p 8 manage_zone.sh -z zbeio01 -n on
Both rewrite an existing attribute and, for a cluster managed zone, push the configuration to the other nodes by themselves, so no separate -r is needed.
Example 3: relocate a SYNC zone, then send an ASYNC one
manage_zone.sh -z zbeio01 -l -h sr02beio manage_zone.sh -z zfoo -S -h sr01prbw -U ocadm -D tank/zones
Example 4: attach an installation ISO, then detach it
manage_zone.sh -z zfoo -a -c debian-12.iso manage_zone.sh -z zfoo -d -c debian-12.iso
Example 5: pass a SAN LUN through to a VM
manage_zone.sh -z zfoo -a -w 618d6dd100f277ffa20bdeaa0000011e manage_zone.sh -z zfoo -L manage_zone.sh -z zfoo -d -w 618d6dd100f277ffa20bdeaa0000011e
The first command resolves the WWN to /dev/rdsk/c0t618D6DD100F277FFA20BDEAA0000011Ed0p0 if that is how multipathing spelled it, hangs it off the last SCSI controller of the zone or off a new one when that controller is full, delegates the device node into the zone and, for a cluster managed zone, pushes the configuration to the other nodes by itself. The VM picks the disk up at its next boot. On sysadm 1.11.6 it writes diskN and diskifN; on 1.13.1 it writes scsi0-targetJ.
Example 6: attach a tray of LUNs from a list
$ cat /root/zfoo-luns.txt
# DG7 data volumes for zfoo
618d6dd100f277ffa20bdeaa0000011e
618d6dd100f277ffa20bdeaa0000011f
# 618d6dd100f277ffa20bdeaa00000120 not migrated yet
618d6dd100f277ffa20bdeaa00000121
$ manage_zone.sh -z zfoo -a -W /root/zfoo-luns.txt
# equivalently, for a handful of LUNs and no file:
$ manage_zone.sh -z zfoo -a -w 618d6dd100f277ffa20bdeaa0000011e \\
-w 618d6dd100f277ffa20bdeaa0000011f \\
-w 618d6dd100f277ffa20bdeaa00000121
Three LUNs are attached in one commit. Had any of them been unmapped on this node, already attached, or over a limit, nothing at all would have been written and the same command could be repeated after fixing the list. Twenty LUNs in the file would fill scsi0 with sixteen and put the remaining four on a freshly created scsi1.
Example 7: attach during a rolling upgrade
OC_LUN_ATTR_MODE=legacy manage_zone.sh -z zfoo -a -w 618d6dd100f277ffa20bdeaa0000011e
Needed only while the nodes straddle sysadm 1.12, when the version probe refuses to choose for the cluster. Pick the form the node that will actually boot the zone can read.
EXIT STATUS
For -b and -s the status is that of the last zoneadm, svcadm or zonecfg command run. -r returns 0 unless staging or the local configuration export failed; a failure on an individual peer is logged and skipped without changing the status.
FILES
ENVIRONMENT
It is read from the environment, so the usual prefix form (OC_LUN_ATTR_MODE=legacy manage_zone.sh ...) is enough and export is not required; a bare assignment on its own line, without export and without being a prefix to the command, sets only a shell variable and the script never sees it. A misspelt name is silent by construction — the script reads the correct name, finds it unset, and falls through to the version probe — so any variable within an edit distance of two of the real name, upper-cased, is reported as having no effect. That covers a wrong, missing or extra letter, a missing or extra underscore, a trailing character and a difference of case only. The run continues with the probed convention.
SECURITY
Every privileged action goes through pfexec, locally and on the peers, under the OmniCube Administration profile from /etc/security/exec_attr.d/omnicube and /etc/security/prof_attr.d/omnicube; that profile is root-equivalent, so grant it only to accounts you would trust with the root password. Zone names, host names, ssh users, dataset names, ISO file names and the ram, vcpus and vnc values are validated before they reach a zonecfg, zfs, zpool or ssh command line, and every dataset name derived by a source-to-target path translation is re-validated. Configuration blobs are written with | pfexec tee rather than a privileged shell, so the profile never has to grant sh.
SEE ALSO
activate_zone_vnc.sh(1), omnicube_utils(3), oc_lock(3), oc_ssh(3), oc_validate(3), vm_tpl.cfg(5), omnicube(7), autocleansnap(8), autosnap(8), autosync(8), create_zone_config.sh(8), create_zone_pool.sh(8), isolate_node.sh(8), shutdown_windows_server(8), sync_pool.sh(8), zone_monitor.sh(8), zone_save.sh(8), zones_srv_monitor.sh(8).
NOTES
-a and -d never take the name of what they act on as an argument: an ISO is named with -c and a LUN with -w. An ISO change does not mark the configuration as changed, so it is not propagated to the other nodes; run -r afterwards for a cluster managed zone. A LUN change does mark it, and is propagated by itself.
The LUN attribute convention is a property of the installed sysadm, not of the zone: upgrading a node past 1.12 leaves every zone's existing diskN/diskifN attributes in place while the launcher stops reading them, so the disks disappear from the guest at its next boot without anything in the configuration looking wrong. -a -w/-W and -L both warn when they see it. The repair is to detach and re-attach each affected LUN, which rewrites it in the current form; there is no in-place conversion.
OC_LUN_ATTR_MODE is also how a test configuration is produced on purpose: on a node running sysadm 1.12 or later,
OC_LUN_ATTR_MODE=legacy manage_zone.sh -z zone -a -w wwn
writes diskN and diskifN even though that node's launcher cannot read them. Everything else about the attach is real — the device node is resolved and checked, mpathadm runs, the configuration is committed and, for a cluster managed zone, pushed to the peers — so the resulting VM will not boot. That is the point of the option, not a side effect, and each use is logged as a warning saying so. The warning about a zone carrying attributes of both conventions is likewise expected here rather than a fault to repair.
An attach that appears to have hung is almost always inside mpathadm modify lu -b none, which the tool reports per LUN precisely so that this is visible. It is safe to wait: nothing is written to the zone configuration until every LUN of the batch has been initialized.
Do not run -a -w/-W against a zone that is going to be moved with -l or -S to a node on the other side of 1.12: the attributes travel with the configuration and will not be understood on arrival.
-p accepts only digits and commas, so the keyword form used in the bhyve template, for instance sockets=1,cores=2,threads=1, cannot be set with this command; edit the vcpus attribute with zonecfg directly, then run -r.
-l and -S are not interchangeable: -l only makes sense for a zone whose storage the target can already import, and rejects nothing but a target equal to the local host, while -S refuses any zone that has an SMF instance. Both are destructive on the source before they are constructive on the target; when a step fails in between, the error message names the exact zonecfg and zoneadm commands that reinstate the zone locally from zonepath/config. After a successful -S, the source datasets and snapshots still exist and must be removed by hand, and the command assumes every dataset of a zone lives below a path component equal to the zone name, aborting when that convention is not met.