Puredisk content router process (spoold) failing to startup.

Problem

The spoold process is not starting. After running a PureDisk services startup "/etc/init.d/puredisk start" the spoold process shows as "failed".

Starting PureDisk ContentRouter: spoold                              failed

Error Message

Checking the /Storage/log/spoold/spoold.log on the content router we can see this error message

August 23 12:47:03 INFO [46992284444128]: Database Manager: initializing
August 23 12:47:03 INFO [46992284444128]: Database Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: Storage Cache Manager: Initialized in 0.44 seconds.
August 23 12:47:03 INFO [46992284444128]: Storage Cache Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: Spooler Cache Manager: initializing
August 23 12:47:03 INFO [46992284444128]: Spooler Cache Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: DataStore Manager: initializing
August 23 12:47:03 ERR [46992284444128]: -1: ds_checkIdentity: identity file /Storage/data/.identity does not exist
August 23 12:47:03 ERR [46992284444128]: 25004: _storeInit: the identity of data store is not valid, spoold cannot con                      tinue.
August 23 12:47:03 ERR [46992284444128]: 26016: Store Manager: Activate failure.

Cause

Checked the /Storage/data directory to see if this .identity file existed. The .identity file should contain the storagepool ID and the contentrouter ID

# ls -al
total 40
drwxr-x---  3 root root 8192 Aug 23 13:03 .
drwxr-xr-x 16 root root 8192 Jan 26  2011 ..
-rwxr-x---  1 root root    8 Jul  1  2010 .cnf
-rwxr-x---  1 root root   56 Jul  1  2010 .cnt
-rwxr-x---  1 root root   56 Jul  1  2010 .path.cnt
drwxrwxrwx  2 root root   96 Aug 23 13:03 deref

As the file does not exist, it can be generated by running the command

/opt/pdcr/bin/spoold --genidentity      

 

We then see that a .identity file exists

# ls -dl .id*
-rw-r--r-- 1 root root 45 Aug 23 14:16 .identity

And the file contains the required information

# cat .identity
[Main]
StoragePoolID = 1
ContentRouterID = 2

Before starting the contentrouter process it should be investigated why this file was missing and why there are not other files in /Storage/data.

Checked the /etc/fstab and /Storage/data is a separate file system to /Storage.

/dev/vx/dsk/PDDG/PDCRVOL /Storage/data          vxfs    suid,_netdev    0 0  

 This file system is not mounted so that is why the .identity file does not exist and why there is no other files in /Storage/data.

Trying to manually mount /Storage/data results in an error

 # mount /Storage/data UX:vxfs mount.vxfs:                                                                             ERROR: V-3-21268: /dev/vx/dsk/PDDG/PDCRVOL is corrupted. needs checking

Checking the /var/log/messages on the content router we can see errors

Aug 22 19:00:55 server1 kernel: vxfs: msgcnt 6 mesg 039: V-2-39: vx_writesuper - /dev/vx/dsk/PDDG/PDCRVOL file system super-block write error
Aug 22 19:00:56 server1 kernel: vxfs: msgcnt 7 mesg 037: V-2-37: vx_metaioerr - vx_logbuf_write - /dev/vx/dsk/PDDG/PDCRVOL file system meta data write error in dev/block 0/752965
Aug 22 19:00:56 server1 kernel: vxfs: msgcnt 8 mesg 031: V-2-31: vx_disable - /dev/vx/dsk/PDDG/PDCRVOL file system disabled
 

 

 

Solution

After making sure the backend storage has no issues an FSCK of the file system should resolve the problem with the file system

/opt/VRTS/bin/fsck -t vxfs  /dev/vx/dsk/PDDG/PDCRVOL

Once complete you should be able to mount the file system

#mount /Storage/data

Then start the Puredisk services

#/etc/init.d/puredisk start

NOTE:- Once the system is back up and running, it may be worth doing a consitenct check in case any data has been lost due to the file system issues.

 


Terms of use for this information are found in Legal Notices.

Search

Survey

Did this article answer your question or resolve your issue?

No
Yes

Did this article save you the trouble of contacting technical support?

No
Yes

How can we make this article more helpful?

Email Address (Optional)