In the year 2010

Another year have passed with lots of changes for Solaris users, some of them good but others quite bad. Most of them are of course related to the Oracle acquisition of Sun which completed January 2010.

The not so good things:

• OpenSolaris is dead
• We no longer have insight into the development of Solaris.next
• We no longer have access to the latest ON source.
• Oracle have lost many brilliant and well known engineers from Sun

Oracle is clearly focusing on Solaris but not in the open kind of way that we go used to with OpenSolaris. As I have written before, this will probably make lots of money for Oracle in the short run, but how can it grow the the Solaris customer base as the OpenSolaris model could?

On the bright side:

• Solaris 11 Express 2010.11, the pre-release of Solaris 11 due next year
• illumos, the community driven ON source gate
• Oracle begun to share some Solaris/SPARC roadmaps this fall
• SPARC T3, 16 cores with great throughput but with limited single thread performance

Solaris 11 Express is in reality a rebranded OpenSolaris 2011.11 release and has a huge amount of new features and enhancements compared to the last OpenSolaris release. The SPARC T3 is a good processor for highly threaded applications and we have seen roadmaps with a T4 coming next summer with three times faster single thread performance.

In 2010 the sparcv9 blog have had over 40 000 visitors counted as "unique" by statcounter. Happy new year everyone!

Solaris 11 Express support repo

If you have a valid support contract with Oracle it is now possible to generate certificates for the Solaris 11 Express support repository.

It's the same process as with OpenSolaris extra and support repositories but the location differs. Login using your My Oracle Account and generate certificates here:

https://pkg-register.oracle.com

VirtualBox 4 Beta on Solaris

Several beta releases of VirtualBox have been released in the last weeks. I had problems with the first two but beta 3 and later works nicely for me on Solaris 11 Express.

New features in 4.0

• New settings/disk file layout for VM portability; see the manual for more information.
• Open Virtualization Format Archive (OVA) support; see the manual for more information.
• VMM: support more than 1.5/2 GB guest RAM on 32-bit hosts
• Language bindings: uniform Java bindings for both local (COM/XPCOM) and remote (SOAP) invocation APIs
• Chipset: added support for the Intel ICH9 chipset with 3 PCI buses, PCI express and Message Signaled Interrupts (MSI)
• Audio: Intel HD Audio is now available as guest hardware, for better support with modern guest operating systems (e.g. 64-bit Windows; bug #2785).
• GUI: redesigned user interface with guest window preview
• GUI: new display mode with downscaled guest display
• Resource control: added support for limiting a VM's CPU time and IO bandwidth.
• Storage: support asynchronous I/O for iSCSI, VMDK, VHD and Parallels images
• Storage: support for resizing VDI and VHD images
• Windows Additions: support for automatically updating the Guest Additions (requires installed Windows Guest Additions 4.0 or later)
• Guest Additions: support for copying files into the guest file system

Update: Whats seems to be the release images are now available:
http://download.virtualbox.org/virtualbox/4.0.0

Two Solaris releases for the T4 in 2011?

Solaris 11 is not the only Solaris release we can expect to see in 2011. The next update of Solaris 10, update 10 or Oracle Solaris 10 8/11 which seems to be what Oracle are aiming for in both name and release date is also coming in 2011. Earlier posts have included some of the expected content but the name and release window recently surfaced.

I found this in a publicly available bug report:

                Oracle Solaris 10 8/11 s10x_u10wos_02 X86
Copyright(c) 2010, Oracle and/or its affiliates. All rights reserved.
                    Assembled 22 November 2010
     Early Access Build for Evaluation and Test Purposes Only.
         NOT SUPPORTED FOR PRODUCTION USE.  Not patchable.

With the 8/11 target it seems that Oracle wants to release the update in time before Oracle World 2011, perhaps to have a new Solaris release to talk about in the sessions but it also seem likely that it will support the upcoming SPARC T4 (Yosemite Falls) processor that could be launched at the same time ( Oracles road maps have pointed to the second half of 2011 and Rick Hetherington stated in the recent interview that it was less than 12 month away ).

Another interesting thing is that in the Oracle roadmap, Solaris 11 was plotted at the same time as the SPARC T4, sometime in the second half of 2011. Could this mean both Solaris 11 and Solaris 10 8/11 will be released at the same time and at Oracle world 2011? Given all the announcement at last Oracle world it would make sense.

Take a look at the slides of John Fowler, Executive Vice President of Systems, page 8 for the roadmap.

This gives some more substance to Fowlers statement that they will continue to produce Solaris 10 updates for some time, the timeframe for the release was probably not very hard to guess but it is interesting that there now seems to be a preliminary name. It looks like next year will be an interesting year for both Solaris and SPARC.

Here is some information on some of the expected content of the update:
Solaris 10 update 10 update

More SPARC T4/T5 information

Oracle has made an interview with Rick Hetherington, Oracle vice president of hardware development available. There is quite a bit of information about the future and development of the SPARC processor, well worth a read.

"Going forward, the SPARC T4 is less than 12 months away. So that's pretty quick. And there's a reason behind that. We wanted to get more single thread performance into the SPARC T-series systems sooner rather than later. So we developed a new core for the SPARC T4 that brings together the combination of throughput performance through threading as well as really high-speed single thread performance. It's really breakthrough technology for us. Referring back to the timeline for development that we have, we developed that core and that technology back in 2006/2007 and we'll be delivering that to the market in 2011.

And then the SPARC T5 will have 16 cores as opposed to the eight cores that we have in the SPARC T4, in 28 nanometer technology. As we go forward with each of these new introductions we'll make sure that the interfaces that deal with memory are common and are contemporary with the sweet spot of memory technology."

Read the whole article here.

Two more well known ZFS developers quit

Previously since the Oracle acquisition the father of ZFS Jeff Bonwick and the co-lead in the project Bill more have left the company.

I know that the ZFS is created by a team but I had missed that another two well known developes also have left Oracle and joined DTrace creator Adam Leventhal at Delphix. This time it is George Wilson and Matthew Arhens.

Delphix welcomes Matt Ahrens and George Wilson:
"George and Matt bring an amazing knowledge of ZFS — the lower, and upper halves respectively — and are also just great engineers who are already contributing tangibly to the the success of Delphix."

Best of luck all three and Delphix. I hope Oracle has or finds people to finish the remaining large development areas of ZFS such as the block pointer rewrite project which I know Matthew have worked with.

Adam Leventhal also quits Oracle

Another bright mind leaves Oracle

ZFS encryption at ${HOME}

A nice addition to easily use of ZFS encryption for home directories will be available in a future version of Solaris 11. This will allow you to have your password as key for your home dataset which is automatically used at login:

"For users with local ZFS storage we want to provide a very simple and as transparent as possible way of using encrypted ZFS datasets. The target for this is laptops and systems with local ZFS storage for the users home directory.

The goal is to provide as seemless as possible a way to have an encrypted home directory and additional encrypted datasets below the home directory. A new PAM module pam_zfs_key.so will be introduced. This module supports only pam_sm_setcred(3PAM) and pam_sm_chauthtok(3PAM), pam_sm_authenticate(3PAM) is provided but always returns PAM_IGNORE.

It assumes that the users login passphrase is also the passphrase used to protect thier ZFS encrypted home directory and will ensure that when users change their password the passphrase used for deriving the wrapping key for their encrypted ZFS home directory is changed as well."

Bugid: 6983112

Solaris 10 update 10 update

Some additional details on the next Solaris 10 update have surfaced, they compliment the list in my previous post. No huge updates, Solaris 10 should not be seeing any ground breaking changes this late in its lifespan but good additions besides the bug fixes.

• IPfilter IPv6 NAT support
• 10Gb Ethernet support for Mellanox ConnectX-2 chipset
• SSH Support for ChrootDirectory and ConnectTimeout
• PSARC/2008/256 Native LDAP standalone tools (Duckwater Phase 0)
• Updated FireFox to 3.6.8+ and Thunderbird 3.1+

The ZFS update should include bits similar to snv_148 which among other things include the following:

• RAID-Z/mirror hybrid allocator
• Missing log import (PSARC/2010/292) (CR 6733267)
• ZIL synchronicity PSARC/2010/108
• read-only import (PSARC 2010/306)

There might also be an update to the bundled samba which is a bit overdue. This was delayed earlied due to change of licenses but that should now have been solved. This might slip again, as it did previously, but it looks like a 3.5+ version of samba will be included.

Update:

I should probably have mentioned that it seems that the name for the release will be Oracle Solaris 10 8/11 which means it is scheduled for a release around August/September 2011. This would be just one or two months before Oracle World so it seems highly likely it will support the expected SPARC T4. This new release cycle with one release per year seems like a better fit for enterprise customers, Solaris 10 9/10 is probably not in many datacenter yet.

Solaris 10 update 10

ZFS Feature/Dist table

For a advanced, low cost, data consistent storage server ZFS still has very little competition. The main challenge today is to choose the correct distribution for your needs. We have S7000/Fishworks and NexentaStor, but both are appliances and cost a fair bit of money, at lest for anything but trivial deployments (>18TB). I put together a table with ZFS features, storage features and cost/support status, included are the larger distributions with Solaris/OpenSolaris kernel today, FreeBSD and OpenSolaris 2009.06 as a reference.

It's not much of a surprise that Solaris 11 Express comes out with the most complete feature set, but it also has a cost of $1000/year for up to 8 cores (1 socket, non Oracle HW) for anything but testing and development. That is cheap for a file server with all the features ZFS provides but for home filers it can be a bit too much.

This table is of course subject to change. Both Nexenta and FreeBSD will surly continue to update ZFS at least to the point were Oracle stopped releasing the source. The available source from Oracle does not include some of the features in the table, for example ZFS crypto and probably a fair amount of bug fixes.

Release	S11 Exp	NCP	FreeBSD	OSOL
Free for use in production	N	Y	Y	Y
Commercial support available	Y	N1	N1	N1
Zpool version	31	26	14	14
ZFS Diff	Y	N	N	N
RAID-Z/mirror hybrid allocator	Y	N	N	N
Improved 'zfs list' performance	Y	N	N	N
Encryption	Y	N	N	N
Multiple vdev replacements	Y	N	N	N
zdb w decomp, checksum, raidz	Y	N	N	N
Low priority resilver	Y	N	N	N
Deduplication	Y	Y	N	N
Tripple-parity raidz	Y	Y	N	N
Log device removal	Y	Y	N	N
Improved scrub stats	Y	Y	N	N
Improved snapshot deleteion performance	Y	Y	N	N
zpool recover	Y	Y	N	N
resilver prefetch	Y	Y	N	N
zpool split	Y	Y	N	N
zil synchronicity	Y	Y	N	N
ZIL	Y	Y	Y	Y
L2ARC	Y	Y	Y	Y
Checksum	Y	Y	Y	Y
Snapshot	Y	Y	Y	Y
Clones	Y	Y	Y	Y
COMSTAR	Y	Y	N	Y
In kernel CIFS/SMB	Y	Y	N	Y

1. No support from sponsoring project or company, may be available elsewhere.

Quick Solaris 11 Express walkthrough

This is a quick command/output walkthrough of some example administrative tasks in Solaris 11 Express. You can just skim thought it or run the command yourself on a test installation. It uses zfs-crypto, deduplication, zones, crossbow, ipadm, zonestat and pkg.

Create a alternate boot environment by cloning the current filesystem state for an quick and easy rollback option:
# beadm create initial

Set up static IP and add a persistent route:
# ipadm create-addr -T static -a local=10.0.10.242/24 bge0/v4
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
bge0/v4 static ok 10.0.10.242/24
lo0/v6 static ok ::1/128

# route -p add default 10.0.10.1
add net default: gateway 10.0.10.1
add persistent net default: gateway 10.0.10.1

Add mail notifications on hardware/FMA and SMF state changes:
# pkg install smtp-notify
# svccfg setnotify -g from-online,to-maintenance mailto:admin@somehost.com
# svccfg setnotify problem-diagnosed,problem-updated mailto:admin@somehost.com

Install some utilities from the network repository:
# pkg install terminal/screen mercurial nmap

Create an encrypted dataset for the secret project:
# zfs create -o encryption=on -o mountpoint=/vault rpool/vault
Enter passphrase for 'rpool/vault': ********
Enter again: ********

Add a dataset with deduplication:

zfs create -o dedup=on -o mountpoint=/export/vbox rpool/vbox

Add a NFS/SMB share area and ignore the ZIL for asynchronous writes in favor of performance(unsafe):
# zfs create -o mountpoint=/export/share rpool/share
# zfs set sharesmb=on rpool/share
# zfs set sharesmb=name=share rpool/share
# zfs set sharenfs=on rpool/share
# zfs set sync=disabled rpool/share

Create a virtual switch with crossbow and two virtual interfaces for zones with bandwidth management and assigned to different CPUs:
# dladm create-etherstub etherstub01
# dladm create-vnic -l etherstub01 vnic_zone01
# dladm create-vnic -l etherstub01 vnic_zone02
# dladm set-linkprop -p maxbw=10M -p cpus=3 vnic_zone01
# dladm set-linkprop -p maxbw=10M -p cpus=4 vnic_zone02

Configure and install zone capped to 50% of a CPU located on ZFS and using the virtual interface:
# zfs create -o mountpoint=/zones -o compression=on rpool/zones

# zonecfg -z zone01
zonecfg:zone01> create
zonecfg:zone01> set zonepath=/zones/zone01
zonecfg:zone01> set ip-type=exclusive
zonecfg:zone01> ad capped-cpu
zonecfg:zone01:capped-cpu> set ncpus=0.5
zonecfg:zone01:capped-cpu>end
zonecfg:zone01> add capped-memory
zonecfg:zone01:capped-memory> set swap=2G
zonecfg:zone01:capped-memory> end
zonecfg:zone01> add net
zonecfg:zone01:net> set physical=vnic_zone01
zonecfg:zone01:net> end
zonecfg:zone01> commit

# zoneadm -z zone01 install
A ZFS file system has been created for this zone.
Publisher: Using solaris (http://pkg.oracle.com/solaris/release/ ).
Image: Preparing at /zones/zone02/root.
Sanity Check: Looking for 'entire' incorporation.
Installing: Core System (output follows)

# zoneadm -z zone01 boot
# zonestat 5

Collecting data for first interval...
Interval: 1, Duration: 0:00:05
SUMMARY                    Cpus/Online: 4/4   Physical: 8063M    Virtual: 11.8G
                    ----------CPU---------- ----PHYSICAL----- -----VIRTUAL-----
              ZONE  USED %PART  %CAP %SHRU  USED   PCT  %CAP  USED   PCT  %CAP
           [total]  0.76 19.2%     -     -  946M 11.7%     - 1960M 16.2%     -
          [system]  0.10 2.54%     -     -  774M 9.60%     - 1816M 15.0%     -
            global  0.66 16.6%     -     -  133M 1.65%     -  113M 0.94%     -
           zone01  0.00 0.00% 0.03%     - 38.1M 0.47%     - 30.0M 0.24% 1.46%

Enjoy the new faster scrub/resilver that should have less of impact on other workloads to the pool. It also has a more detailed output:
# zpool scrub rpool
# zpool status rpool
pool: rpool
state: ONLINE
scan: scrub in progress since Sat Nov 20 02:21:51 2010
3.22G scanned out of 19.0G at 35.8M/s, 0h7m to go
0 repaired, 16.95% done
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0t0d0s0 ONLINE 0 0 0

Check if any updates are available:
# pkg update -n
No updates available for this image.

Oracle blogs on S11 Express features

Lots of Solaris 11 information have been made available after the launch of Solaris 11 Express, including technical blog entries from Solaris engineers. Here are links to the ones I found most interesting:

Darren Moffat on ZFS crypto:
Introducing ZFS crypto in Oracle Solaris 11 Express
Choosing a value for the ZFS encryption property
Assured delete with ZFS dataset encryption
Having my secured cake and Cloning it too (aka Encryption + Dedup with ZFS)
ZFS encryption what is on disk ?

The Observatory:
Upgrading from OpenSolaris 2009.06 to Solaris 11 Express 2010.11

Alan Coopersmith on X11:
X11 changes in the 2010.11 release

Another great resource for Solaris 11 is the Oracle video blog which now hosts the following videos:

• New Security Features in Oracle Solaris 11 Express
• ZFS Features in Oracle Solaris 11 Express
• Solaris 11 Engineering Panel at LISA 10
• What's Great in Solaris 11 Express for Sysadmins
• Solaris 11 Packaging & Installation
• What's Great in Solaris 11 Express for Developers
• DTrace BoF at LISA10
• Oracle Solaris Studio and Oracle Solaris 11 Express
• Solaris 11 Express: Zones

Several of the videos are in multiple parts so I just link to the whole blog:

Oracle Solaris Video

Oracle Solaris Express 2010.11

A little summary of what you can expect the first release of Solaris 11, Solaris 11 Express 2010.11.

There is nothing drastically different from OpenSolaris, Solaris 11 Express is a rebranded later build with fixed bugs and new features, the build is snv_151a. A result of this is that it is possible to upgrade from OpenSolaris to Solaris 11 Express, there is a chapter about that in the release notes here.

The list of new features is long, personally I most look forward to features in ZFS such as crypto, zones integration with crossbow and the new network administration model with ipadm. For my laptop and desktops the upgraded Xorg and Gnome will be most welcome.

The text based installer is now available for both x86 and SPARC, it is something that has been obviously missing for server installations since the first release of OpenSolaris ( there have been previews ).

A long list I put together of new features, drivers etc between the last publicly available build of OpenSolaris (134) to build 146 it makes a nice overview of all the changes that have happened since the almost released 2010.0[23456].
Lots of enhancements, no build available

As I wrote in my last post Solaris 11 Express is already available for download and it is a Oracle supported release:

"Oracle Solaris 11 Express is fully tested and supported on a variety of SPARC and x86-based systems from Oracle, including the Exadata Database machine and Exalogic Elastic Cloud, and other hardware platforms from 3rd party vendors."

I am missing a way of using Solaris 11 Express for personal, non-profit or even startup companies without buying a license agreement. This is most true for people who want to give the Solaris 11 a try but might want to put anything from a personal web-server to the family albums on the same host. The license agreement says you are allowed to use it for the following:

"only for the purpose of developing, testing, prototyping and demonstrating your applications, and not for any other purpose."

Oracle will host a live Solaris 11 Express webcast on the 7th of December here.

A few earlier posts on Solaris 11 Express:
Solaris 11 Express 2010.11 and ZFS
ZFS crypto integrated

The Oracle Solaris 11 Express Overview with link to Downloads and everything you need to get started: Oracle Solaris 11 Express

OpenSolaris build 134b

A bit ironic but as Oracle released Solaris 11 Express, the build that was once supposed to become Opensolaris 2010.05 was also put out the gates. The opensolaris.org/release repository was updated to be used as a platform for upgrading existing OpenSolaris installations to Solaris 11 Express. You can now upgrade to this version anyway:

                      OpenSolaris 2010.05 snv_134b X86
  Copyright (c) 2010, Oracle and/or its affiliates.  All rights reserved.
                           Assembled 28 May 2010

Solaris 11 Express released!

I will post updates to this meanwhile it's available for download here:

www.oracle.com/technetwork/server-storage/solaris11/overview/index.html

What's new documentation is here:
www.oracle.com/technetwork/server-storage/solaris11/documentation/solaris-express-whatsnew-201011-175308.pdf>

Notes from Solaris Summit, LISA 2010

There was a lot of good recapitulation on features that has been available in OpenSolaris that will be part of Solaris 11 as well as some new things.

Fist of all, Solaris 11 Express is coming any day now, they mentioned weeks and days as the timeframe for the release which I suspect will be build snv_151a (or possible another respin). Given the internal tag of the release 2010.11 they should at least be aiming for a November release.

There was a lot of focus on the integration of zones with different core components of Solaris, IPS was discussed in regard to zones and updates and the synergies of integrating zones with crossbow. In the future (post S11 Express) there will also be possible to control storage inside of zones in a good way, if allowed utilities like format and newfs should be usable on devices dedicated to the zone. The storage focus for also include Zones as NFS servers.

Crossbow will be optimized for NUMA localities, a new framework will be available, NUMA I/O and will at least be used by infiniband besides the networking stack.

Solaris 11 will be able to handle the CPU resources dedicated for a zone even for the network utilization. With crossbow threads can be bound to virtual interfaces and these can be kept in sync with the ones dedicated for the zone making the zone an even more isolated environment.

The rewrite of the virtual memory subsystem (VM2.0) is alive and will be delivered incremental in the Solaris 11 lifetime. This should pave way for thinks like virtualized memory which can be dedicated to zones (Really dedicated RAM and swap to the zone, not the capping we have today) as well as power management of memory. A description of VM 2.0 by Blake A. Jones:

"VM2 is a project to redesign the Solaris virtual memory system around modern computer architectures. The core of the current VM system was designed in 1985-86, when Sun's large computers had 4 megabytes of RAM, one simple CPU with a simple MMU, very few disks, and no NUMA or power management. A couple decades of Moore's Law and many many billions of dollars of hardware development later, things look a bit different. Obviously the software has evolved to deal with these hardware changes, and it's a testament to the original design that it's performed for this long. But the VM system has developed a reputation for being hard to understand, as 20 years' of accreted development will tend to do, and having most of the VM interfaces operate on lists of small fixed-size pages has made it hard to do more significant innovation."

There also was some ZFS news, but that has been covered by this blog before: ZFS crypto, RAID-Z/mirror hybrid allocator and some stuff available in the external source before it closed. One thing new was some numbers on performance impact of crypto and speed enhancement with the new raidz allocator:

"Actual costs for ZFS encryption is 7% for random I/O,
and 3% for sequential I/O

ZFS RAID-Z mirror allocator - preliminary data is that
some workloads are 2 to 4 times faster for things like
directory searching"

There is much more interesting stuff in the videos and presentation slides, both are still available here:
blogs.sun.com/video/entry/join_the_live_video_stream

ZFS Crypto integrated
Solaris 11 Express 2010.11 and ZFS
Zones should be able to be NFS servers

Live Solaris 11 streams from Lisa 2010 tonight

There will be a Solaris 11 live stream from LISA 2010 tonight. The topics will include the new packaging system IPS, ZFS, Network virtualization (crossbow, can they still call it that externally?), Solaris zones and security. The selection of speakers looks promising.

Here is the agenda (TZ: PTS, UTC/GMT -8):

• 9:00 - 9:30 am - Introduction to Oracle Solaris 11 Express, by Markus Flierl
• 9:30 - 11:00 am - Image Packaging System, by Bart Smaalders
• 11:00 am - 12:30 pm - Deploying Oracle Solaris 11 in the Enterprise, by Dave Miner
• 12:30 pm - 1:30 pm - LUNCH BREAK
• 1:30 pm - 2:30 pm - Advances in Solaris Networking with Crossbow and Beyond, by Nicolas Droux
• 2:30 pm - 3:00 pm - Oracle Solaris Containers in Oracle Solaris 11 Express, by Dan Price
• 3:00 pm - 3:15 pm - BREAK
• 3:15 pm - 4:15 pm - ZFS Features in Oracle Solaris Express, by Cindy Swearingen
• 4:15 pm - 4:45 pm - New Security Features in Oracle Solaris 11 Express, by Glenn Faden
• 4:45 pm - 5:30 pm - Deploying Applications Using SMF and Other Solaris 11 Features, by Liane Praza
• 5:30 pm - 6:30 pm - Beer and Snacks Reception for ALL

Update: Here is a direct link to the stream: www.ustream.tv/channel/solaris-summit-at-lisa10

Oracle Solaris Summit will be streaming live at LISA 2010!
Large Administration System Admministartion (LISA) 2010

McNealy about Java and Capitalism

Scott McNealy held a keynote at the PostgreSQL database conference. He discussed the Sun acquisition, Java and capitalism. He highlights the points that Sun was trying to do "good" by sharing and working with the community and make money, while Oracle is focuses more on the money part and not so much on sharing.

"Sun was a company of "good capitalists," McNealy said -- implying virtuous as well as effective -- while Ellison is a "great capitalist." The end result is that "he's there and I'm here," meaning Ellison is still running Oracle and McNealy is without a job."

And

""Sharing's not Larry's middle name," McNealy said. But developers can always take the code for open-source projects and fork it into other projects, something he predicted will happen with both OpenSolaris and Java."

Read the full article at computerworld:
Scott McNealy talks Java, Oracle and Larry Ellison

Both Mike and Brendan quits Oracle

The last of the tree DTrace creators are now leaving Oracle, Mike Shapiro, who also was part of the FishWorks core team. Another member of the FishWorks team was Brendan Greg who also resigned from Oracle. Those of you who didn't now about Bredan before probably got to know who he was when he became famous for screaming at disks in a online video.

DTrace and ZFS is the core foundation of the S7000 storage appliance, now the whole team who invented DTrace is gone and so are the two core persons behind ZFS, Jeff Bonwick and Bill Moore.

There are a lot of other great people behind these products, but the ones listed above are rare and extremly talented engineers and have all been part of breaking new ground in their area of expertice. The FishWorks team even created a usable web interface, a rare thing comming from Sun.

Mike Shapiro: End of file
Brendan Gregg: G'Day and Goodbye
Brendan Gregg screaming, Unusual disk latency
FishWorks, Now it can be told

Nexenta OpenStorage summit

Next week Nexenta will be hosting it's first annual OpenStorage summit in Palo Alto, CA. Besides obvious speakers such as Evan Powell, CEO of Nexenta several people from the ZFS/OpenSolaris community will also be speaking:

Bill Moore, Ex-Sun ZFS Co-creator
Jeff Bonwick, Ex-Sun creator of ZFS
Ben Rockwood, director of systems at Joyent.

Nexenta is as you probably know an OpenStorage vendor which builds it's storage appliance on top of ZFS. They are probably part of the reason Oracle cut the cord on the OpenSolaris source since their product competed with Oracles own OpenStorage appliances, which also are built on ZFS. Nexenta plans to continue to develop the ZFS/OpenSolais source and are sponsors of the illumos project which is the de-facto development gate of the OpenSolaris source now.

It will be interesting to see how they plan to handle the fork of ZFS now that Oracle have continued their development internally, ZFS have gained bug fixes but more importantly new features in newer versions of the on-disk format. If Nexenta plans to enhance ZFS to a degree that requires new version of the pool the two implementations will no longer be compatible.

Nexenta will fall behind Oracle Solaris/Oracle OpenStorage in terms of ZFS features, at least for a while. Oracle has plenty of source already created for ZFS which is yet to be integrated in their Solaris Next source, one of them is the BP rewrite project which has been in the making for years, another is ZFS on-disk encryption which has already been integrated into the internal Solaris gate at Oracle

Will Nexenta sit out and hope for the ZFS source to be released after Solaris 11 (late next year at the earliest) or will the make larger enhancements by their own. Both scenarios are possible while i doubt Nexenta will sit and await for Oracle. The ZFS code already released is stable and contains features not yet integrated into Nexenta products. New development is also possible, especially since several key persons behind ZFS have left Oracle and there are other developers, especially with a company found the development.

In the end this will affect all users of ZFS which are not Oracle customers, Nexenta and illumos are probably where future free ZFS development will be done. All users who want's a free OS with the latest possible ZFS bits are going to be using Nexenta Core or OpenIndiana or some other distribution based on the illumos source.

Nexenta announces first summit
Nexenta announces final agenda and sponsors for its first annual OpenStorage summit

Zones should be able to be NFS servers

NFS service in local zones have been frequently requested since the introduction of zones in Solaris 10, the change request for this even predates Solaris 10 by two years, it was filled in 2003. I've earlier mentioned that some works was being done to this including PSARC/2010/208. Now it seems that this has been implemented and will be integrated into internal build 154 of Solaris Next. This is hopefully in time for the first release of Solaris 11 Express which is expected next month.

Update: I was a bit quick on this one, build 154 should close in the end of next month, that does not work very well with a November release of Solaris Express, so it might be delayed or more probably it will not be part of the first Express release.

Zones should be able to be NFS servers, bugid 4964859
NFS Instances with NFS shares.
PSARC/2010/208

Solaris 11 Express 2010.11 and ZFS

Besides ZFS data set encryption the upcoming Solaris Express release (Which seems to be tagged 2010.11) will contain several enhancements to ZFS besides dataset encryption that was mentioned earlier:

• RAID-Z/mirror hybrid allocator
"Normally mirror vdevs contain mirrored data, RAID-Z vdevs contain RAID-Z data, etc. However, for latency-sensitive metadata, we can use a mirrored layout across the children of a RAID-Z vdev. This ensures that such metadata can be read in a single I/O."


• Support for decompression, checksumming and raidz in zdb -R will make it much easier to debug ZFS errors on disk or just to learn more about the on-disk format of ZFS.


• Performance improvements for listing ZFS filesystems.

The downside of this is that these are features only present in Oracles internal gate and not other distribution besides Solaris 11/Express and the S7000/FishWork appliances can use them until and if Oracle releases the source after Solaris 11 is released.

I hope the Solaris 11 Express license will allow me to use Solaris 11 for my private storage needs, S7000 are nice and deployed at work but still a bit too expensive for my datacenter at home ;)

6977913 RAID-Z/mirror hybrid allocator
6757444 want zdb -R to support decompression, checksumming and raid-z

ZFS crypto integrated

Support for encrypted ZFS datasets have been integrated into Solaris Next Development, build snv_148. This means that there is a good chance it will be available with the first release of Solaris 11 Express which is expected later this year.

This feature have been in the works for years with the original feature request filed in 2003. This will be a welcome addition for ZFS users with sensitive data and/or laptops.

After this only one major component is missing from ZFS, the block pointer rewrite feature. It will make it possible to encrypt/compress/deducplicate existing data in the pools as well as shrink pools and defragment them without the need to move the data out of the pool and in again.

Update:
As noted I failed to include a reference, here is bugid 4854202.

ZFS on disk encryption support
Darren in ZFS crypto video interview
ZFS crypto pushed to next year

FishWorks 2010.Q3 release

A new major update to the FishWork appliance software have been released, 2010.Q3. This is the first release to support the new S7000 appliances released at Open World. It also contains bug fixes and a few enhancement, nothing exiting but it includes new Infiniband administration, better AD support, NDMP stream support and routing enhancements.

These are areas with bug fixes according to the release notes:

• device resilvering following failure and replacement
• snapshot management and deletion
• remote replication
• upgrade, factory reset, and rollback
• Active Directory interoperability
• shadow migration

ak-2010.08.17.1.0 Release Notes
FishWorks software updates

Another bright mind leaves Oracle

I can't say it comes as a surprise but another great mind is leaving Oracle. This time it's Jeff Bonwick, the man behind the slab kernel memory allocator in Solaris and more recently (together with others) ZFS. It is indeed disturbing to see that so many of the innovators from Sun, who stayed with the company thought terrible financial times are now leaving.

This means that most high profile persons behind Fishworks (S7000 appliance), DTrace and ZFS have left the red ship. Jeff Bonwick, Adam Leventhal, Bryan Cantrill and Bill Moore. There are of course many good people still at Oracle, but the people I have been following seem to be on the move.

Best of luck Jeff, I'm sure you will continue to break new ground somewhere else.

Free Your Mind,How a Small, Persistent Team Created a Revolutionary File System
Sun Engineer Jeff Bonwick is New Sun Fellow
Now it can be told (Slab allocator)

More Solaris 11 Express information

The first release of Oracle Solaris 11 Express will be later this year with a full release of Solaris 11 in 2011. There is not much more information on what this release will contain but it is probably a later build of OpenSolaris rebranded for Oracle/Solaris 11.

A interesting note is that Solaris 11 will be powering the new Exadata X2-8 and in the new cloud-in-a-box Exalogic .

Oracle Outlines Next Major Release of Oracle Solaris

New Fishworks release at OpenWorld

Server new products in the S7000 line of the ZFS based OpenStorage products from Oracle was released today. The smallest is the 7120 that comes with configuration from 12TB. The most interesting addition is the 7420 which comes in configurations up over 1PB with optional clustering and caches up to 1TB of RAM and 4TB of Flash. The X4470 or something similar is the Sun server what fits the specification of both the 7420 and the 7720 (which seems to be a in rack cluster version). It features 512GB of memory per controller and can support up to 32 cores just as the 7420. This makes my 7310 feel very even smaller. I guess this is the "Sun T4" I have been referencing earlier.

This is just various hardware configurations running the same OpenStorage software as before, we will see if there is some new features coming in an update to the software.

Oracle Unveils Next-Generation Sun ZFS Storage Appliance Product Line
Sun ZFS Storage 7720 Appliance

LDOM 2.0 released at Open World

Oracle today released LDOM 2.0 as Oracle VM Server for SPARC 2.0. This new released features better resource and power management but most importantly it now finally supports dynamic reconfiguration of memory. It should also support more flexible migration of domains in cooperation with Solaris (Cooperative guest migration) but it does not, as expected, seem to be the full live migration that we are eagerly awaiting. Oracle Solaris 11 Express is also mentioned as a possible OS for a guest domain, no news that it will work but interesting that it's mentioned. An the new SPARC T3 processor is of course supported in this new release.

Oracle Unveils Oracle VM Server for SPARC 2.0

SPARC T3 released at Open World

Oracle today released the SPARC T3 processor and four new systems ranging from the SPARC T3-1 with 1 CPU, 16 cores, 64 threads to the SPARC T3-4 with 4 CPUs 64 cores, 512 threads. No surprises, it the 16 core successor to the UltraSPARC T2(+) which we have been expecting since at least year, enhancements to the built-in crypto support and more cores but the same frequency and single-thread performance.

The minimal supported release of Solaris 10 is 10/09 which is good, I was afraid they would require the latest 9/10 update.

Update:
The minimal supported release seems to have changed in all documentation change to "Solaris 10 09/10", by wihch i assume they mean 9/10. This new way of writing the release name with a leading zero in the month part of the name makes it even harder to decode the name than just having two releaes named 9/10 and 10/09 and it does not match the name in /etc/release:

Oracle Solaris 10 9/10 s10s_u9wos_14a SPARC


SPARC T3 Processor
Oracle Unveils SPARC T3 Processor and SPARC T3 Systems
SPARC T3-4 SERVER Datasheet(PDF)
It's a UNIX system! New and Improved CPU support

Oracle Open World 2010

The first Open World since Oracles acquisition of Sun is here. Oracle likes focus their releases around Open World and we have already seen a new version of Solaris 10, 9/10. I guess that they will release SPARC T3, LDOM 2.0 (Or Oracle VM Server for SPARC) and hardware/software updates to the S7000(FishWorks) the next few days. If we are lucky we might also get some more information on Solaris 11 Express.

I will post as soon as something interesting surfaces.

The evolution of Solaris 10 zones

Solaris zones or Solaris containers dependent on where you work and how you use it have been available in since the initial release of Solaris 10 in 2005. It has worked well for isolating workloads and creating virtual execution environments inside a single kernel with minimal overhead. There have however been a few bumps along the way, especially when it comes to patches and upgrades. With the new Solaris 10 9/10 I feel that most of the problems have been solved so here is a summary of the road behind us.

The upgrade scenario for zones was all but forgotten at the time of the first Solaris 10 release and the first update only supported upgrades when booting from DVD and no live upgrade support.

Sun however realized that this was quite a problem for both them and their customers, so project ZULU was created to solve the problem (Zones Upgrade Live Upgrade, PSARC/2006/167).

ZULU was delivered with Solaris 10 update 4 which was the first release to support real upgrades of zones including live upgrade, but ZFS was still not supported as underlying filesystem for zones and live upgrade did not know how to handle such zones.

ZFS is a key component for easy live upgrade of zones. With zones on ZFS there is no need to copy all OS/Zone instances to a separate filesystem, possible requiring additional disks and overhead of administration. Live upgrade creates a ZFS clone of the needed filesystems and uses them for the upgrade. The support for Zones on ZFS was delivered with Solaris 10 10/08 (Update 6).

Live upgrade was only part of the solution, if you where unable to use live upgrade or if you where forced to have customer applications down even with live upgrade you where still in trouble. The largest problem was the time it took to update a system with many zones, it could take days(!) to update a system with, lets say 30 zones on it. The upgrade/patching process was sequential and on top of that is was very time consuming. This has today been fixed by a couple of enhancements, we now have zones parallel patching that enables several zones to be patched in parallel. The package system have also been updated with "Turbo charged SVR4 packages" making it faster, especially for zone installations and upgrades.

Lastly there is the "update on attach" function making it possible to move zones to a updated machine (or detach the zones and upgrade the current host) then have them updated in parallel when attached to the system with the new OS/patch level. The "update on attach" functionality was delivered in two steps, the first one made sure the zone was in the minimum supported state for running on the host. The second step which came with the latest update of Solaris delivers something more like what most customers would want, all packages are updated which makes it more like an upgrade.

In short, you are implementing zones today, use Solaris 10 9/10, put all zones on ZFS and use Live Upgrade/Update on attach. This will make it easy to upgrade while minimize both the time for the upgrade and the downtime.

Short history of Zone features in Solaris updates:

Solaris 10 11/06 (Update 3)
Zone renaming
Zone move and clone
zone attach/detach
Privileges in zone configuration

Solaris 10 8/07 (Update 4)
Upgrades, Live upgrades (ZULU)
IP Instances (dedicated NIC/separate TCP/IP stack)
Resource setting for memory/CPU

Solaris 10 5/08 (Update 5)
Dry run of zone migration (zoneadm -n)
CPU caps for zones

Solaris 10 10/08 (Update 6)
Update on attach
Default router in shared stack
Full ZFS support for zones

Solaris 10 10/09 (Update 8)
Turbo-Charged SVR4 Packaging
Zones Parallel Patching

Solaris 10 9/10 (Update 9)
Zones P2V (Physical to Virtual)
Host ID Emulation
"Upgrade on attach"

Jerry's blog Zones P2V
Patching zones goes zoom
PSARC/2009/173 Turbo-charging SVr4 package install

OpenIndiana announced

Today project OpenIndiana was announced, it is continuation of OpenSolaris created by the community. It's currently pretty much OpenSolaris build 147 but in the near future it will be based on the illumos gate.

It's nice to see a distribution that is similar to OpenSolaris not everyone wants to go the illumos/debian way like Nexenta and OpenIndiana aims to be a free drop-in replacement for Solaris 11/Oracle Solaris Express. It can be used for free in production unlike the new Solaris 10/Solaris 11 licensing from Oracle. OpenSolaris installation should also be able to upgrade to OpenIndiana.

The slides from the announcement are available here:
http://dlc.openindiana.org/tmp/slides.pdf

Iso Images are also available, but this is the first bits they ever release and they call this release a bit "messy" so use it with caution:
http://dlc-origin.openindiana.org/isos/147/

Solaris 10 9/10 released

I have had several posts about Solaris 9/10 this year and today it was finally released. I'm happy to report that my predictions regarding the content was very accurate. I'we included links to my earlier post, but in short it contains a large ZFS update, several new features for Solaris zones, JDK 1.6 and along overdue update to several desktop tools such as Firefox and Thunderbird.

Oracle also released Solaris Cluster 3.3 and Solaris Studio 12.2 today.

Oracle Solaris 10 9/10 What's new

My earlier post on the release:

Solaris 10 9/10 and second zfs refresh

First hints on Solaris 10 update 9

Desktop updates for Solaris 10 update 9

Solaris 10 update 10

There is a another update planned for Solaris 10 after the yet to be released Solaris 10 9/10. It is still very unclear and probably not fully decided what this update will include, but i'we found indications on the following:

• RDSv3 for use with Oracel RAC
• PSARC/2007/587 Volo Low Latency Socket Framework
• PSARC/2010/108 zil synchronicity
• Adobe reader for X86 version 9.3.X
• Lightning Thunderbird extension for Oracle Beehive

With the zil synchronicity update it seems likely that the next update, just as the last few, will get a large backport of ZFS code from Solaris Next, but with some features turned off, deduplication will be still be disable in update 10. This would release will probably also include support for the new SPARC T processors with better single thread performance, releasing them without having an well established operating system supporting it would probably not make much sense. Since update 9 is not even released yet but planned for this fall update 10 will probably be delivered mid next year or towards Oracle World 2011. I would not be surprised if this will be the last update to Solaris 10 before moving on to Solaris 11.

John Fowler mentioned the upcoming Solaris 10 updates in the systems webcast:

"We will continue to produce Solaris 10 updates for some time for customers who are used to and have deployed Solaris 10 and want to take advantage of new hardware platforms. Comming up with new product introductions this year and then extending in the previous years"

Oracle pulls ON source cord

Oracle have now pulled the plug on the ON source updates, the external hg repository will no longer be updated by Oracle (or anyone else), all Oracle Solaris development will be internally and code only released when a Solaris 11 release is made. The last complete build tag became onnv_147 which closed a few days ago.

May god have mercy on their souls.

This will change my habits a bit, everyday went through the changes in the ON gates to see what have happened. But this might not be the and of days, Garrett D'Amore write about this and the affects on Illumos in his blog. He states that from now on ON and Illumos will go different ways but is very enthusiastic about it. I really hope the Illumos will get momentum and attract enough engineers to stand on it's own. There must be a free Solaris alternative or it will become an Enterprise only option with no community involvement.

I would expect that binary distributions based on the latest source available will become available from some place quite soon. A newer version than the snv_134 which contains a few nasty bugs is long overdue.

The new place to look for changes in the open ON source:
http://www.illumos.org/projects/illumos-gate/repository

Service auto discovery in future Solaris

While looking at changes for the future packaging system in Solaris I stumbled on a putback that included pybonjour in the pkg(5) gate. Apparently Solaris of the future is going to be using zeroconf for auto discovery of packages repositories. Looking through other OpenSolaris project it seems like this and NWAM also will be using auto discovery. OpenSolaris have had support for mDNS discovery since build 72 but the pkg use-case is what got me interested.

This will allow clients on the same network to discover mirrors of the repository and use them to ease the load on the primary repository. The mirroring service is easily enabled:

# svcadm enable application/pkg/dynamic-mirror

From pkg.depotd(1M):

"The pkg depot is also able to act as a mirror server for local client images from pkg(5). This allows clients that share a subnet on a LAN to mirror their file caches. Clients may download files from one another, thereby reducing load on the package depot server. This functionality is available as an alternate depot service configured by smf(5). It uses mDNS and dns-sd for service discovery."

From pkg(1):

"mirror-discovery
(boolean) Mirror-discovery tells the client to discover
link-local content mirrors using mDNS and DNS-SD. If this is
set to True, the client will attempt to download package content
from mirrors it dynamically discovers. To run a mirror that
advertises its content via mDNS, see pkg.mdnsd(1m)."

# pkg set-property mirror-discovery True

Network Auto Magic (NWAM) will include service auto discovery of DNS and naming services:
http://hub.opensolaris.org/bin/view/Project+nwam/service-discovery

Adam Leventhal also quits Oracle

It seems that Oracle is no place for talented high profile engineers, Adam Leventhal is also leaving Sun. Adam was also part of the team that created DTrace and latest Fishworks together with Bryan Cantrill who left a few weeks ago. The most known persons from the Fishworks team has their blogs linked from the dtrace.org site, even Mike Shapiro and Brendan Gregg which are still at Oracle. Nice to see that they keep together even when two of them have left Oracle.

I'm sad too see him leave Oracle but I wish him the best in whatever he decides to do. I can only hope that he in some way will continue to be involved with Solaris. Maybe hi swears his allegiance to Illumos just as as Bryan Cantrill did (He is VP of Engineering at Joyent which have stated they will be involved with Illumos). Maybe he even joins Mike at Joyent, it seem like a good fit for creative people, especially ex-Sun (I meet the Joyent guys a year ago over a lunch, fantastic people and a very interesting company).

The new home for Adams blog with the "Leaving Oracle" entry:
http://dtrace.org/blogs/ahl/

ZFS crypto lifesign

Aside from BP-rewrite, encryption is probably the most awaited ZFS feature, but there have been sparse information regarding the status of both projects for about a year.

The ZFS crypto project which are going to deliver on-disk encryption for ZFS have now at least showed a small life sign. This is after a long time of radio silence from Darren, Sun and Oracle regarding the ZFS crypto project since May last year when "PSARC/2009/443 ZFS Crypto Updates" was published. Now there are references to a new PSARC, but it's not public so we have only the name to to look at: PSARC/2010/329 ZFS Crypto additional keysource types.

The last crypto releated PSARC from laste year, which was public:
PSARC/2009/443 ZFS Crypto Updates

zonestat implemented

We are not left out in the dark yet, the gates are still open and we can see changes made to the OpenSolaris source, the source that is going to be used for Solaris 11.

Today PSARC/2010/29 zonestat was implemented, it provides enhanced observability for Solaris zones. prstat -Z can provide a nice overview of all zones on a system but it does not tell you anything about any resource capping of memory or CPU resources. There have been a perl-script with the same name available for some time which is able to extract and present a nice summary of zones and their capping and usage. This is not only a reimplementation in C, it is also architectures the new observability feature into three layers, a collection daemon, an API and a command line utility. This case has requested patch binding and it should not be hard or include any significant risk to backport it to a Solaris 10 update.

Examples from the PSARC:

Example 1: Summary of cpu and memory usage over a 5 second interval:
    # zonestat 5 1
    SUMMARY
                -----CPU------------- ----PHYSICAL--- ----VIRTUAL----
           ZONE USED %PART %CAP %SHRU USED  PCT  %CAP  USED  PCT %CAP
        [total] 9.74   30%    -     - 7576M  23%    - 11.6G  24%    -
       [system] 0.28  0.8%    -     - 6535M  19%    - 10.4G  21%    -
         global 9.10   28%    -     -  272M 0.8%    -  366M 0.7%    -
      kodiak-ab 0.32  1.0%    -     -  256M 0.7%    -  265M 0.5%    -
      kodiak-dp 0.00  0.0%    -     - 77.6M 0.2%    - 71.1M 0.1%    -
kodiak-gjelinek 0.00  0.0%    -     - 58.7M 0.1%    - 59.3M 0.1%    -
     kodiak-edp 0.00  0.0%    -     - 53.0M 0.1%    - 58.9M 0.1%    -
 kodiak-johnlev 0.00  0.0%    -     - 51.9M 0.1%    - 57.4M 0.1%    -
  kodiak-jordan 0.00  0.0%    -     - 51.7M 0.1%    - 56.8M 0.1%    -
   kodiak-steve 0.00  0.0%    -     - 51.5M 0.1%    - 56.2M 0.1%    -
   kodiak-susan 0.00  0.0%    -     - 48.9M 0.1%    - 55.7M 0.1%    -
kodiak-batschul 0.00  0.0%    -     - 48.5M 0.1%    - 49.5M 0.1%    -
 kodiak-garypen 0.00  0.0%    -     - 46.3M 0.1%    - 49.5M 0.1%    -
     kodiak-rie 0.00  0.0%    -     - 22.7M 0.0%    - 49.4M 0.1%    -

PSARC/2010/291 zonestat

Name change for UltraSPARC T3

The UltraSPARC T3 processor have had it's name changed to just SPARC T3. Just something Oracle marketing came up with I suppose.

The source changes are here: http://hg.genunix.org/onnv-gate.hg/rev/91026ef504cf

OpenSolaris dead as a distribution

A supposedly internal memo from Oracle has been posted to the opensolaris-discuss list. The most important part of this is probably that it finally states that OpenSolaris as a distribution is dead, Oracle will no longer make OpenSolaris available in binary form as a release or in bi-weekly development builds. The source will for most parts continue to be open, but will not be pushed out in realtime as it does today, it will be posted after a Solaris 11 release has been made available. This way Oracle will always be first in line to productize their latest innovations and fixes. neither will Oracle continue to post most of the PSARC externally. There will however be options for partners to gain early access to source and binaries from Oracle. Solaris 11 will be made available in an Solaris 11 Express form as a preview to the next release of Solaris 11. It will be free to use for developers and has an optional support plan.

As am enterprise customer the message is great, to know that Oracle really believes and invests in Solaris and that Solaris 11 will be made available early. The availability of the source after a release is also very important. I also feel that Oracle genuinely are going to make Solaris even better and are willing to put in the resources to do so. Looking at all the new features available in OpenSolaris since Solaris 10, Solaris 11 will be a huge leap forward for Solaris.

As someone involved and personally interested in the OpenSolaris community this is not very good. I will be cut of from the day-to-day changes in the source and no free OpenSolaris distribution will be available to attract new customers outside of the large enterprises. I think there will be some other distribution that will try to take the place of OpenSolaris, but with the access to the source cut, except for snapshots from releases it will not be as interesting as the weekly builds have been. It would also stall months behind Solaris 11 as it has to wait for the release to be made available.

If Oracle just would make Solaris 11 available at a lower price with obviously less support commitment it might come something good out of this even for businesses with smaller budgets. I think this is really something Oracle must do, otherwise the adoption rate of Solaris will decline drastically. It would neither undercut the sales of the enterprise support contracts, they will still be sold, mission critical deployments are not run with patch or mail-only support options. I've been involved with several smaller startups which love Solaris and like to use it, but neither need or can afford the support contracts for Solaris today, it would be sad if there was no open option for them to continue to use Solaris for a, for them, reasonable price.

So in short, I think that this will make Solaris even better for the large customers, Oracle will put much effort in keeping Solaris the most capable and advanced operating system available. But smaller companies, startup and hobbyist will have a harder time to adopt Solaris 11. I will continue to use Solaris 11 express for my private servers if possible, but I can obviously not be as involved as before since I get cut off from frequent builds and changes to the source.

The full memo is available here.

Webcast available and SPARC roadmap

The webcast is now available at Oracles website

While the presentation for the future SPARC roadmap stretched to 2015 a few things in the next year or so caught my attention. First a CPU with double throughput will be delivered this year. This is obviously the UltraSPARC T3 with double amount of cores than the current generation of T-processors, the UltraSPARC T2. Early next year the M-Series SPARC64 processors will get some additional speed, probably with slightly higher clock-frequency. But next year is when it gets interesting, they plan to deliver three times the single-thread performance in a new generation of T-processors! It must be a whole new core in this chip since the current, more simple cores would have to be clocked far over 4GHz to achieve this. Looking into 2012 a new generation of M-servers are planned, with 8-64 cores, providing 6x the throughput and 1.5x the single-thread performance. The roadmap goes on until 2015 with several times the performance for both T and M series of servers, but we will hopefully have plenty of time to get back to that in the next years.

All this sound good, but the thing I would look forward to the most is the 3x boost of single-thread performance for T-servers, making it possible to use LDOM technology for a much wider range of workloads. By that time we should also have live migration of LDOMs between T-servers available.

I would love to dig deeper into the details in this but it's my last days of vacation and I have an early flight tomorrow morning.

Oracle Systems Strategy Update Fowler

Oracle confirms Solaris 11

John Fowel, executive vice president of system, now at Oracle have disclosed some a timeframe for the Solaris 11 release. It should be released in 2011 and contain many of the new features and enhancements that has been implemented in OpenSolaris. It should even contain a superset of what is currently in OpenSolaris, with new technologies created after the Oracle acquisition.

I've previously discussed that Oracle might be focusing on Solaris 11 instead of OpenSolaris, and that seems to have been correct. They does not mention any new release of OpenSolaris in this web event.

This also confirms the name "Solaris 11" for the next Solaris release which always officially have been Solaris next or Solaris.next since marketing wants to have the options open for the name of the next release. Many of you probably remember the change from Solaris 2.7 to Solaris 7.

I will write more when I and if more information is made available, the web event will be made available online later.

You can read the whole article from Serverwatch here.

Update: Eweek has some more information regarding this while awaiting the webcast. Oracle plans to double the performance of SPARC every year through 2015: Oracle Outlines SPARC, Solaris 11 Plans.

Read-only zpools implemented

Read-only zpools have now been implemented in the source as proposed in PSARC/2010/306.

Previously individual filesystem have could be mounted as read-only, but now the whole pool can be imported this way:
# zool import -o readonly=on zpool01

When importing a pool with read-only, no writes will be performed to any of the devices in the pool, they are even opened read-only.

From the PSARC:
"When the 'readonly' property is 'on' (read-only) the following actions are performed:

1). pool is loaded but transaction processing is disabled
2). all filesystems and zvols are mounted in read-only mode
3). any intent-log replays are deferred (any pending synchronous
writes will be replayed once the pool is imported read-write)

The following restrictions apply when a pool is read-only:

1). Attempts to set additional zpool-level properties during import
are ignored.
2). All filesystem mounts are converted to include the 'ro'
mount option.
3). Additional mount options may be set on a temporary basis.

A pool which has been imported read-only can be restored back to read-write by exporting and reimporting it."

PSARC/2010/306 Read-only ZFS pools

ZFS diff

A really nice feature have been implemented for ZFS, which allows you to list all file changes between two snapshots of a ZFS filesystem. The PSARC was filled in March but it has now been implemented. It will have a simple syntax just like diff(1):
# zfs diff zpool01/fs01@snap01 zpool01/fs02@snap02

With output in this format:

M       /myfiles/
M       /myfiles/link_to_me   (+1)
R       /myfiles/rename_me -> /myfiles/renamed
-       /myfiles/delete_me
+       /myfiles/new_file

From the PSARC:
" There is a long-standing RFE for zfs to be able to describe
what has changed between the snapshots of a dataset.
To provide this capability, we propose a new 'zfs diff'
sub-command. When run with appropriate privilege the
sub-command describes what file system level changes have
occurred between the requested snapshots. A diff between the
current version of the file system and one of its snapshots is
also supported.

Five types of change are described:

o File/Directory modified
o File/Directory present in older snapshot but not newer
o File/Directory present in newer snapshot but not older
o File/Directory renamed
o File link count changed

PSARC/2010/105 zfs diff

Lots of enhancements, no build available

Since build 134 of OpenSolaris which once was destined to become the OpenSolaris 2010.1H release quite a lot have changed. If there was a release based on that build today it would feel a bit old giving all the changes now available. I took a look on some of the larger or in my opinion more noticeable changes and made a list. It makes you want to get your hands on a Solaris Next developmen build soon, or at least update the home rolled builds to b146 once it is finished. The development efforts does not seem to have decreased, it's a shame we still don't have any public builds to test all these new features.

This list is just a overview of the enhancements, there have been over 1300 putbacks containing over 2200 bugfixes including many new features. This is only the OS/Net consolidation with the core part of OpenSolaris not including PKG, X, JDS or the many free software packages available for installation from the package repositories.

PSARC/2010/014 Open Source elxl replacement
PSARC/2010/042 increase max TCP_INIT_CWND
PSARC/2008/532 NWAM Phase 1
PSARC/2009/576 pam_krb5 PKINIT support
PSARC/2010/004 Logical Domains Information API and ldminfo
PSARC/2010/067 Interim modernization updates
PSARC/2010/045 x86gentopo enumeration of direct attached SATA
PSARC/2009/354 Always on / no reboot Solaris Audit
PSARC/2009/364 dlstat and flowstat
PSARC/2010/041 USB CDC ECM driver
PSARC/2010/062 increase number of realtime signals
PSARC/2009/689 Audio DDI Simplifications
PSARC/2009/534 SMB/CIFS Standalone DFS
PSARC/2010/009 Modified ZFS passthrough-x ACL inheritance
PSARC/2010/032 EC and SHA2 for KMF
PSARC/2010/013 SMF Early Manifest Import
PSARC/2009/104 Hot-Plug Support for ACPI-based Systems
PSARC/2009/306 Brussels II - ipadm and libipadm
PSARC/2009/421 Open Fabrics User Verbs (OFUV) - primary kernel components
PSARC/2009/019 SAS Management Protocol library
PSARC/2010/102 ikeadm dump algs
PSARC/2010/085 IPoIB Administration Enhancement
PSARC/2010/029 Improved ACL interoperability
PSARC/2010/101 in.iked preshared key file extensions
PSARC/2010/048 zfs-based ndmpd backup
PSARC/2010/119 "Console User" assignment, logindevperm and virtual console update
PSARC/2010/043 Reliable Datagram Service v3
PSARC/2009/377 In-kernel pfexec implementation.
PSARC/2010/108 zil synchronicity
PSARC/2010/127 ipadm hostmodel property
PSARC/2010/155 sshd(1M) PAM Service name options
PSARC/2010/174 TMPDIR support for Solaris tar
PSARC/2009/646 bd - generic block device driver
PSARC/2010/028 BSD List Interfaces
PSARC/2010/181 PRIV_SYS_RES_BIND privilege
PSARC/2010/106 DTrace TCP and UDP providers
PSARC/2010/151 new socket options for TCP timers
PSARC/2010/083 Kerberos Profile API
PSARC/2010/188 PKCS#11 URI parser for libcryptoutil
PSARC/2010/177 KMF Certificate Name mapping extensions
PSARC/2010/144 lofi(7D) in non global zones
PSARC/2009/590 Socket Filter Framework
PSARC/2007/477 Drivers registering devids
PSARC/2010/222 Relaxed Type Requirements for SMF Profiles
PSARC/2010/215 Add libstmf interfaces to set/get default stmf properties
PSARC/2009/505 IRM Framework Extension(s)
PSARC/2009/430 Default system CA (X.509) Certificates
PSARC/2010/195 PKCS11 KMS Provider
PSARC/2009/042 max-processes rctl
PSARC/2010/240 Allow ranges for the 'cpus' datalink property
PSARC/2010/166 layer-3 net properties for exclusive-IP zones
PSARC/2010/239 OFUV Userland Interface
PSARC/2010/235 POSIX 1003.1-2008 *at(2) syscalls
PSARC/2010/241 userattr
PSARC/2010/111 exclusive IP for s10c
PSARC/2010/269 KMF Certificate Validation Enhancements
PSARC/2010/164 interfaces for basic install network configuration
PSARC/2010/113 Boot Block Downgrade Avoidance
PSARC/2010/234 IBTF 2010.Q2 Enhancements
PSARC/2009/617 Software Events Notification Parameters CLI
PSARC/2009/618 snmp-notify: SNMP Notification Daemon for Software Events
PSARC/2009/619 smtp-notify: Email Notification Daemon for Software Events
PSARC/2010/278 FMA/SMF integration: instance state transitions
PSARC/2010/279 Modelling panics within FMA
6914369 enable s10brand on system booted under xvm
6889694 COMSTAR kstat to track IO statistics at the ITL
6655655 Single phase COMMIT processing for NFS3 and NFS4 server
6830088 Fast reboot support for SPARC platforms
6871356 Sata framework should support INQUIRY VPD page 0xB1
6782858 Solaris needs Broadcom 5718 network support
4404421 RFE: tar command -z and -j support
6202362 Solaris tar can't unpack files created with GNU tar
6922272 SATA framework does not handle >2TiB disks
6939349 RFE: Update ksh93 to ast-ksh.2010-03-09
6923815 Implement FMA functionality in qlge driver
6889322 Add NLP2020 PHY support to nxge
6874797 Solaris needs to support I/O subtree reparenting
6945131 SMF should support multiple administrative profiles
6951174 Intel microcode 2010-02-09 update
6923890 Add FMA to SiliconImage si3124 sata driver
6946100 Add machine check and performance counter support for new member of Nehalem family
6937229 Solaris needs to support BCM5717 and BCM5724
6953601 improve logical unit creation time for zvol backed logical units
6910752 mpt_sas driver performance can be improved
6494473 ZFS needs a way to slow down resilvering
6917066 zfs block picking can be improved
6932959 ZFS allows too many devices to be faulted
6716117 ZFS needs native system attribute infrastructure
6675946 'zpool status' should show the progress of resilvering for individual disk.
6668666 zpool command should put a bootblock on a disk added as a mirror of a root pool vdev
6733267 Allow a pool to be imported with a missing slog
6959846 DMU traverse prefetch size should be a global tunable
6866610 Add SATA TRIM support
6908227 pyzfs needs to move to python 2.6

If any of the links point to nowhere or in the case of PSARC:s empty or almost empty directories there are no public information available besides the source.

Backporting

Last night i spend a few hours backporting recent changes from the ON gate to my own gate that is based on ONNV_134. I wanted to get a build that is as close to OpenSolaris b134 as possible while pulling fixes and a few enhancements for ZFS, COMSTAR, SATA and NFS. I went quite easy, the main problem was that i did want to get as close to the latest ZFS bits without including the changes that removed the iSCSI daemon (not because I will use it but it was part of 134). With a few modifications to the patches i now have a ONNV_134 build with a much more current ZFS. My plan is to test this for a while and then use it for my ZFS storage nodes at home, where of course ZFS is the most important part. There have been quite a few fixes for ZFS since February when build 134 closed.

This is obviously only something I will be playing with until Oracle decides to release a Solaris version based on a more current snapshot of the OpenSolaris source. Private builds and independent distributions can be both fun and useful but does not go trough the amount of testing as the "official" releases does, that kind of fragmentation decreases the testing for all distribution.

In the case of ZFS i like to use OpenSolaris and preferably a more current one. It is the growing ground of Fishworks, the software part of the S7000 series which I use for some production environments. The ZFS code in those should go trough a fair amount of testing both by Oracle and all of the thousands of enterprise deployments.

All this said, times goes by and with no binary release in sight a community effort to build and deliver binaries for the community for testing, preparation for the next release and recruit new users might be needed. But it would be sad if we ended up divided between an Oracle paid distribution and a community one with no involvement by Oracle other than providing almost all of the source and new features of the ON code, which they could decide to stop delivering or only deliver parts of it. But all this is in the hands of Oracle.

Continued Solaris support on HP/Dell

After some time of uncertainty Oracle have have now announced that Oracle Solaris will continue to be supported on HP and Dell system:

Oracle press release:
"Oracle today announced Dell and HP will certify and resell Oracle Solaris, Oracle Enterprise Linux and Oracle VM on their respective x86 platforms.

Customers will have full access to Oracle’s Premier Support for Oracle Solaris, Oracle Enterprise Linux and Oracle VM running on Dell and HP servers. This will enable fast and accurate issue resolution and reduced risk in a company’s operating environment.

Customers who subscribe to Oracle Premier Support will benefit from Oracle’s continuing investment in Oracle Solaris, Oracle Enterprise Linux and Oracle VM and the resulting innovation in future updates."

Notification enhancements

A common request from smaller shops or private users of Solaris is that they want email notifications when something is wrong with their system. This is most common with ZFS filers, they might not have any surveillance systems implemented but want to be sure to get a notification if something goes wrong with a zpool. A quick fix to this is of course a small script which can dump zpool or FMA errors over email but it is not a standardized or complete solution.

Now there have been a large putback in OpenSolaris that among other things delivers this functionality. FMA faults and SMF events can with these change be configured to deliver notifications over mail or SNMP. It also enables the fault management daemon in local zones so that they can deliver SMF state changes from within the zone. Using this framework you can also configure any of your own SMF services to deliver state change notification over any of these two protocols.

PSARC/2009/617 Software Events Notification Parameters CLI
PSARC/2009/618 snmp-notify: SNMP Notification Daemon for Software Events
PSARC/2009/619 smtp-notify: Email Notification Daemon for Software Events
PSARC/2010/225 fmd for non-global Solaris zones

SATA Trim command in B146

The SATA TRIM command have been implemented in the OpenSolaris source, build 146. This can be used by filesystems to prevent performance degradation over time on SSD:s by giving the garbage collector feedback on which blocks are no longer in use. This is bugid 6866610. It it still not implemented in ZFS but this is the ground stone for making that possible and there is a feature request for that also 6957655.

Looking into the future the similar SCSI UNMAP command would be of great use for thin provisioning, both for thin devices located on a ZFS LUN and for ZFS residing on a thinly provisioned LUN for example in a SAN.

The later is described in this bug report 6913905.

New ZFS recovery option

Currently if you loose a non-redundant log device in your zpool it becomes impossible to import, there is no way to exclude the log device and import the rest of the pool. There are however light on the horizon, PSARC 2010/292 proposes a solution to this problem with a new switch for the zpool import command.

From the PSARC:
"This fast-track introduce a new command line flag to the
'zpool import' sub-command. This new option, '-m', allows
pools to import even when a log device is missing. The contents
of that log device are obviously discarded and the pool will
operate as if the log device were offlined."

PSARC 2010/292 zpool import despite missing log

Update
This has now been implemented in the source, the changeset is here.

Good luck Bryan

There are many good engineers working with Solaris, but a few of them get noticed a bit more. Bryan Cantrill is one of those and sadly he is now leaving Sun/Oracle. Bryan is the inventor of DTrace which he created together with Mike Shapiro and Adam Leventhal and he have even kissed a girl. Once upon a time he was requited to Sun Microsystems by Jeff Bonwick, the father of ZFS. The last few years he was a vital part of the FishWorks team used in the S7000 storage series that uses DTrace to produce graphical analytics of performance data.

I wish Bryan the best whatever he decides to do. I will keep an eye out for his next endeavor.

The new home of Bryan's blog which also holds his last entry from the Sun/Oracle blog: dtrace.org/blogs/bmc

Bryan's profile at Sun Labs: Bryan Cantrill - Sun Labs

NFS Instances/Zones with NFS shares

There is a PSARC awaiting approval for providing on of the most long standing enhancements requests for Solaris Zones: NFS Shares from within a local zone. This functionality will not be limited to NFS and will initially also support CIFS. It seems to be planned for Solaris Next only, so no backport to Solaris 10.

From the PSARC:
"Patch binding is requested; however, there are no plans to backport any of
the proposed changes.

The proposed changes are needed to support an NFS server in a non-global
zone; however, the changes are not specific to NFS. They would also be used
when enabling any file sharing protocol (ie. CIFS) server for non-global zones."

PSARC/2010/2080