SNFS/Xsan: Quantum SNFS metadata controller and Xsan client compatibility chart

In a previous life, I designed and built many SANs based on Xsan (I believe I started with Xsan 1.3). I then migrated to looking after SANs based on SNFS, either from 3rd party vendors, or Quantum.

I believe that the age of Fibre Channel is long over (although SNFS also works on Infiniband if I recall correctly). The advantages of block-level access have been eclipsed by the much higher bandwidth with Ethernet, at a fraction of the cost.

However, I want to share some of my documentation with the world. This is done under an Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.

The information has been collected from Apple support articles (current and obsolete ones), ADIC’s and Quantum’s StorNext documentation, and personal experience.

Every Xsan 2.0 and above client has been included. Maybe one day I will add Xsan 1.x releases for historical purposes

Xsan 20.0 Xsan 5.0.1 Xsan 5 Xsan 4.1 Xsan 4 Xsan 3.1 Xsan 3 Xsan 2.3 Xsan 2.2 to 2.2.2 Xsan 2 to 2.1.1
11.0.1 10.13, 10.14, 10.15 10.12 10.11 10.10 10.9 10.8 10.7 10.6 10.5
SNFS 7.0.x
SNFS 6.4.0
SNFS 6.3.x
SNFS 6.2.x ? ? ?
SNFS 6.1.x ? ? ? ?
SNFS 6.0.5, 6.0.5.1, 6.0.6 ? ? ? ?
SNFS 6.0, 6.01, 6.0.1.1 ? ? ? ?
SNFS 5.4.x ? ? ? ?
SNFS 5.3.2.x
SNFS 5.3.1
SNFS 5.3.0
SNFS 5.2.2
SNFS 5.2.1
SNFS 5.2.0
SNFS 5.1.x
SNFS 5.0.x
SNFS 4.7.x
SNFS 4.6
SNFS 4.3
SNFS 4.2.1
SNFS 4.2.0
SNFS 4.1.1 to 4.1.3
SNFS 4.0 to 4.1
SNFS 3.5.x
SNFS 3.1.2 to 3.1.5
SNFS controller and Xsan client compatibility chart.

There are some caveats with some of the supported configurations. Some releases were originally marked by Apple as incompatible and then reverted. In the same way, some configurations that were originally marked as working were then updated as not compatible.

Double-check official documentation before any deployment.

I hope you find this table useful. There are some additional Xsan curiosities I will post in the future.




macOS : Removing caches and temporary files in macOS

The easiest way is using an application like Onyx.

You can check the amount of space taken by the files using du.

$ du -ch /path
# du -ch /path

Before using any deletion command you can check with ls what exactly is going to be deleted.

$ ls /path
# ls /path

When using rm enable verbose mode (-v). Don’t delete the root folders, just the files contained in them. If you have a trailing /** it will ensure you don’t have any mishaps.

$ rm -rfv /path/**
# rm -rfc /path/**

As with everything related to deleting files. Be careful.

User & App Caches

List files.

[Logged user caches]
$ du -ch ~/Library/Caches/
$ du -ch~/Library/Logs/

[All users caches, requires sudo]
# du -ch /Users/*/Library/Caches/
# du -ch /Users/*/Library/Logs/

Delete files.

[Current user caches]
$ rm -rfv ~/Library/Caches/**
$ rm -rfv ~/Library/Logs/**

[All users caches, requires sudo]
# rm -rfv /Users/*/Library/Caches/**
# rm -rfv /Users/*/Library/Logs/**

Check also your home folder for hidden configuration files and folders. This tends to be filled with quite a lot of old stuff.

$ ls -la ~

Trashes

On shared systems deleting trashes might be of help.

You will be emptying other people’s trash. Ask for permission!

List files

[Current user trash]
$ du -ch ~/.Trash/

[All users trash]
# du -ch /Users/*/.Trash/

Delete files.

[Current user trash]
$ rm -rfv ~/.Trash/**

[All users trash]
# rm -rfv /Users/*/.Trash/**

System Caches

List files.

# du -ch /Library/Caches

Delete files.

# rm -rfv /Library/Caches/**

/private/var

Manually deleting these folders isn’t recommended. Although I haven’t checked in detail if there are any exceptions.

macOS will clean the folders during boot. Just restart the system.

If folders aren’t cleared, boot in Safe Mode and then restart again.

/var/vm

In this location sleep images used to be stored. They can be quite bulky and can be deleted.

Temporary files

Usually located in /tmp, but you can also use the system variable $TMPDIR.

/tmp/ can be cleared safely.By definition, anything stored in that folder will be deleted sooner or later anyway.

Browser Caches

These would have already been deleted by deleting the applications caches earlier. They are listed here for completion.

Safari

Listing files.

$ du -ch ~/Library/Caches/com.apple.Safari/Cache.db

# du -ch /Users/*/Library/Caches/com.apple.Safari/Cache.db

Deleting files.

$ rm -rfv ~/Library/Caches/com.apple.Safari/Cache.db

# rm -rfv /Users/*/Library/Caches/com.apple.Safari/Cache.db

Browser Caches (Chrome)

~/Library/Caches/Google/Chrome/
~/Library/Application Support/Google/Chrome/Default/Application Cache/

Listing files.

$ du -ch "~/Library/Caches/Google/Chrome/" "~/Library/Application Support/Google/Chrome/Default/Application Cache/"

# du -ch "/Users/*/Library/Caches/Google/Chrome/" "/Users/*/Library/Application Support/Google/Chrome/Default/Application Cache/"

Deleting files.

$ rm -rfv "~/Library/Caches/Google/Chrome/" "~/Library/Application Support/Google/Chrome/Default/Application Cache/"

# rm -rfv "/Users/*/Library/Caches/Google/Chrome/" "/Users/*/Library/Application Support/Google/Chrome/Default/Application Cache/"

Browser Caches (Firefox)

Listing files.

$ du -ch ~/Library/Caches/Firefox/Profiles

# du -ch /Users/*/Library/Caches/Firefox/Profiles

Deleting files.

$ rm -rfv ~/Library/Caches/Firefox/Profiles

# rm -rfv /Users/*/Library/Caches/Firefox/Profiles

DNS Caches

$ sudo dscacheutil -flushcache ; sudo killall -HUP mDNSResponder

# dscacheutil -flushcache ; killall -HUP mDNSResponder



macOS: Application stealing focus

A few weeks ago I had an issue with a macOS system which focus kept being stolen by an unknown application. All the windows were displaying as inactive so it had to be a background process.

A StackExchange user called medmunds had adapted a script from another post, that seems to have been modified from another from some Apple forum. Isn’t the Internet great?

His script very quickly displayed the culprit. Very useful indeed.

#!/usr/bin/python

try:
	from AppKit import NSWorkspace
except ImportError:
	    print "Can't import AppKit -- maybe you're running python from brew?"
    print "Try running with Apple's /usr/bin/python instead."
    exit(1)

from datetime import datetime
from time import sleep

last_active_name = None
while True:
    active_app = NSWorkspace.sharedWorkspace().activeApplication()
    if active_app['NSApplicationName'] != last_active_name:
        last_active_name = active_app['NSApplicationName']
        print '%s: %s [%s]' % (
            datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
            active_app['NSApplicationName'],
            active_app['NSApplicationPath']
        )
    sleep(1)



SNFS/Xsan: Changing a volume name

In the past it used to be a very straightforward process. You would rename the volume configuration file and run cvfsck.

With newer versions if you try to do that you will get an error message.

To make the name change you can use the cvupdatefs command.

If you have more than one volume running, the below instructions will allow you to rename one volume while the rest are still running, minimising downtime.

Stopping the file system

Stop the file system

# cvadmin -e 'stop oldname_volume1'

Check that it hasn't failed over.

# cvadmin -e 'select'

If it has failed over to another server just run the first command again until the volume you want to rename isn't running.

Check the filesystem

# cvfsck -j oldname_volume1
# cvfsck -nvvvvv oldname_volume1

If errors are shown at the above you need to fix them. Ideally you want to dump inode information before any big repair but that is for another article.

You can fix the errors with:

# cvfsck -vvvvv

Run the above command until there are no errors shown.

Changing the volume name

You can now change the volume name.

# cvupdatefs -R newname_volume2 oldname_volume1

Update the name of the volume in fmslist.

SNFS
# vim /usr/cvfs/config/fsmlist

Xsan
# vim /Library/Preferences/Xsan/fsmlist

In Xsan you need to push the changes to the second metadata server.

# xsanctl pushConfigUpdate

In Xsan you might need to check that the name isn’t referenced in any other configuration file (/Library/Preferences/Xsan), but you can run grep and see where you might need to make changes.

Also in Xsan, if needed, copy the configuration file to the second metadata server. Be aware that Xsan Admin does very often fail to make a good copy of the configuration to the second server. Run a file checksum on both ends and copy the volume configuration file manually if it doesn’t match.

This issue with Xsan Admin will in the best case not allow a volume to fail over, and in the worst case, cause data loss.

In SNFS/Linux you should check for any references of the old name in /usr/cvfs/config/

Also in SNFS/Linux, make sure that the changed files are also updated on the second metadata server.

Remounting the filesystem

In Xsan you don’t need to issue a new profile for the clients to mount the new volume. Just mount it once from the CLI and it will automount on restart:

# xsanctl mount newname_volume2

On Linux clients update entries in /etc/vstab or /etc/fstab to automount the volume on boot.

On Windows clients you will need to use the SNFS configuration tool to mount the newly named volume.

For HA environments Quantum has published an article with the above steps for their systems.




macOS: Spotlight results not showing applications

Sometimes if your system restarts unexpectedly Spotlight stops working as expected. In this particular case it won’t show any Applications on the results despite the settings on System Preferences.

This will happen even if you disable and re-enable Spotlight through the GUI. The solution is do it via the CLI.

Stop Spotlight indexing:

# mdutil -a -i off
[...]
/Volumes/Untitled:
2020-01-26 10:59:42.019 mdutil[13212:548120] mdutil disabling Spotlight: /Volumes/Untitled -> kMDConfigSearchLevelFSSearchOnly
	Indexing disabled.

Stop Spotlight:

# launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

Start Spotlight:

# launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

Start Spotlight indexing:

# mdutil -a -i on
[...]
/Volumes/Untitled:
	Indexing enabled.

The results should now show applications.




Linux: Configure locale and keyboard layout when remotely accessing from a Mac

At work I have to remote into several different Linux systems from a Mac and there is always the pain of having to handle different keyboard layouts if using Synergy or VMs.

The conversion from a Mac keyboard layout doesn’t translate correctly when the Linux system has the keyboard configured as a PC.

First find out what the current configuration is.

$ localectl

The output in a British system is:

  System Locale: LANG=en_GB.UTF-8
                 LANGUAGE=en_GB:en
     VC Keymap : n/a
     X11 Layout: gb
      X11 Model: pc105
    X11 Options: terminate:ctrl_alt_bksp

Note that the last line might not show in Ubuntu systems.

Unfortunately if you are accessing from a Mac this layout will not work and basic things like |pipe| will not be easy to find.

You can edit the locale and keyboard layout of the system you are accessing so the mapping matches your Mac keyboard layout.

The ideal configuration should be like this:

  System Locale: LANG=en_GB.UTF-8
                 LANGUAGE=en_GB:en
     VC Keymap : n/a
     X11 Layout: gb
      X11 Model: macintosh
    X11 Variant: mac
    X11 Options: lv3:alt_switch

Ubuntu 18.04 / Ubuntu 20.04 / Debian 9 / Debian 10

Edit the following file:

/etc/default/keyboard

With these entries:

XKBMODEL="macintosh"
XKBLAYOUT="gb"
XKBVARIANT="mac"
XKBOPTIONS="lv3:alt_switch"

The XKBOPTIONS I have here are for Synergy to keep the Control and Alt keys on the Mac working the same on the Linux systems. You might not need or want it. Just remove it from the commands if that is the case.

You can also do a text GUI configuration of the keyboard with:

# dpkg-reconfigure keyboard-configuration

If your environment isn’t in English the menus won’t be either. You can force the language output of the application launched to be in the default one. That would be English in most cases. The same command as above but forcing the output to be in English:

# LC_ALL=C dpkg-reconfigure keyboard-configuration

An internet search of LC_ALL will show you more languages and options if needed.

CentOS 6

I have only been able to change the keyboard to a British layout, the Mac layout doesn’t seem to work.

Edit the following file:

/etc/sysconfig/i18n

with the following:

LANG="en_GB.UTF-8"
SYSFONT="latarcyrheb-sun16"
SUPPORTED="en_GB.UTF-8:en_GB:en"

Or type the following command:

# loadkeys uk

You might need to restart if editing i18n, but the change should be automatic with loadkeys.

CentOS 7

Edit the following file:

/etc/locale.conf

with the following:

LANG="en_GB.UTF-8"

Or type the following command:

# localectl set-locale LANG=en_GB.UTF-8

Set the keymap:

# localectl set-x11-keymap gb macintosh mac lv3:alt_switch

In CentOS 7 it isn’t necessary to reboot, the above command automatically loads the key mappings.

CentOS 8

The same commands used for CentOS 7 fail. I suspect that there is a file or folder with the keyboard mappings that has been moved. It might be a bug or a deprecated feature.

Just in case I opened a bug report with CentOS.

Regardless, the following will set the locale and the keyboard mappings correctly:

# localectl set-locale LANG=en_GB.UTF-8
# localectl set-keymap gb-mac

There is no need to reboot.




macOS: macOS installer fails with “No packages were eligible for install”

Having to re-install recently mac OS High Sierra 10.13 from a USB flash drive I got stuck getting the following error message:

macOS could not be installed on your computer.

No packages were eligible for install. Contact the software manufacturer for assistance.
Quit the installer to restart your computer and try again.

This happened repeatedly and I had recently used that same USB flash successfully.

It seems that the macOS installers will only work within a very defined time period after release. You can overcome this by changing the date during the installation.

After booting up and before selecting the volume where you want to install macOS you need to open the Terminal and change the date with the date command. Then proceed with the installation.

I am sure this issue happens with other macOS releases but I haven’t tested them all. It is a safe bet to change the date to a date not far from the original release date.

These examples below set the date to one day after the official release date of each version.

[OS X 10.11 El Capitan]
# date -u 1001000015

[macOS 10.12 Sierra]
# date -u 0921000016

[macOS 10.13 High Sierra]
# date -u 0925000017

[macOS 10.14 Mojave]
# date -u 0925000018

Feedback and confirmation if any of the above didn’t work for you is appreciated.