Mongodb service won't start

Question

I am not sure why, and the service was working fine just yesterday, all of a sudden I can not get MongoDB to start up.

[root@purr ~]# service mongod start
Starting mongod: Thu Aug 29 23:26:07.550 
Thu Aug 29 23:26:07.550 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
Thu Aug 29 23:26:07.550 
about to fork child process, waiting until server is ready for connections.
forked process: 5078
all output going to: /var/log/mongo/mongod.log
ERROR: child process failed, exited with error number 100
                                                           [FAILED]

I looked in /var/log/mongo/mongod.log and the following:

***** SERVER RESTARTED *****


Thu Aug 29 23:29:05.874 [initandlisten] MongoDB starting : pid=5118 port=27017 dbpath=/var/lib/mongo 32-bit host=purr.ipetfindr.com
Thu Aug 29 23:29:05.874 [initandlisten]
Thu Aug 29 23:29:05.874 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Thu Aug 29 23:29:05.874 [initandlisten] **   32 bit builds are limited to less than 2GB of data (or less with --journal).
Thu Aug 29 23:29:05.874 [initandlisten] **   Note that journaling defaults to off for 32 bit and is currently off.
Thu Aug 29 23:29:05.874 [initandlisten] **   See http://dochub.mongodb.org/core/32bit
Thu Aug 29 23:29:05.874 [initandlisten]
Thu Aug 29 23:29:05.874 [initandlisten] ** WARNING: You are running in OpenVZ. This is known to be broken!!!
Thu Aug 29 23:29:05.875 [initandlisten]
Thu Aug 29 23:29:05.875 [initandlisten] db version v2.4.6
Thu Aug 29 23:29:05.875 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673
Thu Aug 29 23:29:05.875 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Thu Aug 29 23:29:05.875 [initandlisten] allocator: system
Thu Aug 29 23:29:05.875 [initandlisten] options: { config: "/etc/mongod.conf", dbpath: "/var/lib/mongo", fork: "true", logappend: "true", logpath: "/var/log/mongo/mongod.log", pidf$
**************
Unclean shutdown detected.
Please visit http://dochub.mongodb.org/core/repair for recovery instructions.
*************
Thu Aug 29 23:29:05.875 [initandlisten] exception in initAndListen: 12596 old lock file, terminating
Thu Aug 29 23:29:05.875 dbexit:
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to close listening sockets...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to flush diaglog...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to close sockets...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: waiting for fs preallocator...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: closing all files...
Thu Aug 29 23:29:05.875 [initandlisten] closeAllFiles() finished
Thu Aug 29 23:29:05.875 dbexit: really exiting now

How should I go about fixing this as restarting the server, and service does not seem to work.

ERROR after repair

Not sure what to do it is saying something about error couldn't open file /var/lib/mongo/local.ns terminating

***** SERVER RESTARTED *****


Fri Aug 30 04:49:43.041 [initandlisten] MongoDB starting : pid=3300 port=27017 dbpath=/var/lib/mongo 32-bit host=purr.ipetfindr.com
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Fri Aug 30 04:49:43.041 [initandlisten] **   32 bit builds are limited to less than 2GB of data (or less with --journal).
Fri Aug 30 04:49:43.041 [initandlisten] **   Note that journaling defaults to off for 32 bit and is currently off.
Fri Aug 30 04:49:43.041 [initandlisten] **   See http://dochub.mongodb.org/core/32bit
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] ** WARNING: You are running in OpenVZ. This is known to be broken!!!
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] db version v2.4.6
Fri Aug 30 04:49:43.041 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673
Fri Aug 30 04:49:43.041 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Fri Aug 30 04:49:43.041 [initandlisten] allocator: system
Fri Aug 30 04:49:43.041 [initandlisten] options: { config: "/etc/mongod.conf", dbpath: "/var/lib/mongo", fork: "true", logappend: "true", logpath: "/var/log/mongo/mongod.log", pidf$
Fri Aug 30 04:49:43.058 [initandlisten] couldn't open /var/lib/mongo/local.ns errno:13 Permission denied
Fri Aug 30 04:49:43.058 [initandlisten] error couldn't open file /var/lib/mongo/local.ns terminating
Fri Aug 30 04:49:43.059 dbexit:
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to close listening sockets...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to flush diaglog...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to close sockets...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: waiting for fs preallocator...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: closing all files...
Fri Aug 30 04:49:43.059 [initandlisten] closeAllFiles() finished
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: removing fs lock...
Fri Aug 30 04:49:43.059 dbexit: really exiting now

Answer 1

After running the repair I was able to start the mongod proccessor but as root, which meant that service mongod start would not work. To repair this issue, I needed to make sure that all the files inside the database folder were owned and grouped to mongod. I did this by the following:

1. Check the file permissions inside your database folder
   1. note you need to be in your dbpath folder mine was /var/lib/mongo I went to cd /var/lib
   2. I ran ls -l mongo
2. This showed me that databases were owned by root, which is wrong. I ran the following to fix this: chown -R mongod:mongod mongo. This changed the owner and group of every file in the folder to mongod. (If using the mongodb package, chown -R mongodb:mongodb mongodb)

I hope this helps someone else in the future.