Important Notes

  1. The Oxide CLI, Go SDK, and Terraform Provider have been updated for API enhancements described under New Features. Please be sure to upgrade.

  2. The OxQL metrics endpoints have been moved from /v1/timeseries to /v1/system/timeseries.

Installation

Oxide Computer Model 0 must be installed and configured under the guidance of Oxide technicians. The requirement may change in future releases.

Upgrade Compatibility

Upgrade from version 11 is supported. We recommend shutting down all running instances on the rack before software update commences. Any instances that aren’t stopped for software update are transitioned to the failed state when the control plane comes up. They can be configured to start automatically with auto-restart policy or they can be started manually by the user.

All existing setup and data (e.g., projects, users, instances) remain intact after the software update.

New Features

Resizing stopped instances

Instance vCPU count and memory size can now be changed in place when instances are stopped. Prior to version 12, users had to detach disks, delete the instance, recreate it with the desired size, and reattach the disks. Use the web console, oxide instance update CLI command, or PUT /v1/instances API endpoint to change instance size. See the Update Instances section in the user guide for more information.

Instance resize

PSU firmware updater

The power shelf controller has been enhanced to support the automatic and autonomous updates of the PSU firmware on the MWOCP68-3600 series.

Performance tuning

  • Intra-VPC network I/O

  • Disk I/O

    • Only send flushes when Downstairs is idle; send Barrier otherwise (crucible#1505)

    • Remove delay-based backpressure in favor of explicit queue limits (crucible#1515)

    • Remove remaining IOPS/bandwidth limiting code (crucible#1525)

    • Remove ackable_work; ack immediately instead (crucible#1552)

Web console

Full console changelog

Bug fixes and other enhancements

  • Switch link FEC is no longer required for new links and will be configured automatically based on the transceiver type (omicron#6992)

  • MGS and Wicket should connect to the second power shelf, labeled as "PSC 1", if one is available (omicron#6993)

  • Correctly install nat/router internet gateway tags during create to rectify sporadic instance connectivity issues. (omicron#7172)

  • Reconfigurator now supports the replacement of the ClickHouse zone and the instantiation of multi-node ClickHouse cluster

  • SAML Provider API docs should mention the private key RSA PKCS#1 format requirement (omicron#7088)

Firmware update

  • Murata PSU firmware update

Known Behavior and Limitations

End-user features

Feature AreaKnown Issue/LimitationIssue Number

Image/snapshot management

Disks in importing_from_bulk_writes state cannot be deleted directly. The procedure for unsticking a canceled disk import can be used as a workaround.

Image/snapshot management

Image upload sometimes stalls with HTTP/2 on Firefox.

Image/snapshot management

The ability to modify image metadata is not available at this time.

Instance orchestration

Instance hostname validation has been strengthened. Instances with a now-invalid hostname will fail to start, though they can still be listed and viewed. If the disks attached to them are valuable, they may be detached from the invalid instances, and re-attached to a new instance. The invalid instance may be deleted at that time.

Instance orchestration

Instances fail to start when one of the switch zones is unavailable.

Instance performance

The tsc clocksource is treated as unreliable by guest, resulting in its fallback to use substantially slower timestamp syscalls. A workaround for this issue can be found in the Troubleshooting Guide.

VPC internet gateway

Changing a silo’s default IP pool causes some instances to lose their outbound internet access. This is due to a mismatch between the pool containing the instances' external IP (which are allocated from the new default pool) and the pool attached to the system-created internet gateways (which are linked to the old pool during creation time). See the Troubleshooting Guide for some possible options for restoring instance outbound connectivity.

VPC routing

Subnet update clears custom router ID when the field is left out of request body.

VPC routing

Network interface update clears transit ips when the field is left out of request body.

-

Telemetry

VM instance memory utilization and VPC network/firewall metrics are unavailable at this time.

-

Operator features

Feature AreaKnown Issue/LimitationIssue Number

Access control

Device tokens do not expire.

omicron#2302

Control plane

Sled and physical storage availability status are not available in the inventory UI and API yet.

omicron#2035

Control plane

The built-in test silo named "default-silo" has resource quotas and should be removed.

omicron#5731

Control plane

Operator-driven software update is currently unavailable. All updates need to be performed by Oxide technicians.

-

Control plane

Operator-driven instance migration across sleds is currently unavailable.

-

Control plane

New instances cannot be created when the total number of NAT entries (private-to-external IP mappings) in the system exceeds 1024.

omicron#6939

User management

User offboarding from the rack is not supported at this time. Apart from updating the identity provider to remove obsolete users from the relevant groups, operators will need to remove any IAM roles granted directly to those users in silos and projects.

omicron#2587