独角兽：[致命]无法创建计时器线程：资源暂时不可用
Unicorn: [FATAL] Failed to create timer thread: Resource temporarily unavailable
我们已经使用 Gitlab 作为我们的版本控制系统。
但是独角兽服务今晚崩溃了
with ps -ef | grep unicorn 我看不到任何关于独角兽的进程
当我想使用 nohup bundle exec unicorn -c config/unicorn.rb -E production -p 3000 2>&1 >dev/null &
启动它时
它说
E, [2015-04-21T05:32:49.215133 #27488] ERROR -- : reaped #<Process::Status: pid 14751 SIGTRAP (signal 5)> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.234264 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.258069 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.274698 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.291256 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.307713 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.324021 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
我找到了 unicorn 崩溃时的日志
#
# Fatal error in ../src/platform-linux.cc, line 825
# CHECK_EQ(0, result) failed
# Expected: 0
# Found: 11
#
==== C stack trace ===============================
1: V8_Fatal
2: v8::internal::Thread::Start()
3: v8::internal::Sampler::Start()
4: v8::internal::Isolate::Init(v8::internal::Deserializer*)
5: v8::internal::Snapshot::Initialize(char const*)
6: v8::V8::Initialize()
7: v8::Locker::Initialize(v8::Isolate*)
8: rr::Locker::setupLockAndCall(int*, unsigned long)
9: rr::Locker::doLock(int, unsigned long*, unsigned long)
10: ??
11: ??
12: ??
13: ??
14: ??
15: rb_class_new_instance
16: ??
17: ??
18: ??
19: rb_obj_instance_eval
20: ??
21: ??
22: ??
23: rb_yield
24: rb_ary_each
25: ??
26: ??
27: ??
28: ??
29: ??
30: rb_class_new_instance
31: ??
32: ??
33: ??
34: rb_yield
35: rb_ensure
36: ??
37: ??
38: ??
39: ??
40: ??
41: ??
42: ??
43: ??
44: ??
45: ??
46: ??
47: ??
48: ??
49: ??
50: ??
51: rb_yield
52: rb_ary_each
53: ??
54: ??
55: ??
56: ??
57: ??
58: rb_catch_obj
59: ??
60: ??
61: ??
62: ??
63: rb_yield
64: rb_ary_each
65: ??
66: ??
67: ??
68: rb_iseq_eval
69: ??
70: ??
71: ??
72: ??
73: ??
74: rb_iseq_eval_main
75: ??
76: ruby_exec_node
77: ruby_run_node
78: ??
79: __libc_start_main
80: ??
E, [2015-04-21T04:47:33.135673 #27488] ERROR -- : reaped #<Process::Status: pid 29410 SIGTRAP (signal 5)> worker=3
I, [2015-04-21T04:47:33.154016 #29410] INFO -- : worker=3 ready
error: cannot fork() for git-upload-pack: Cannot allocate memory
X-Accel-Mapping header missing
E, [2015-04-21T04:47:33.135673 #27488] ERROR -- : reaped #<Process::Status: pid 29410 SIGTRAP (signal 5)> worker=3
I, [2015-04-21T04:47:33.154016 #29410] INFO -- : worker=3 ready
error: cannot fork() for git-upload-pack: Cannot allocate memory
X-Accel-Mapping header missing
#
# Fatal error in ../src/platform-linux.cc, line 825
# CHECK_EQ(0, result) failed
# Expected: 0
# Found: 11
当我使用 nohup bundle exec unicorn_rails -l 0.0.0.0:3000 -E production -c config/unicorn.rb 时，一切正常，有人能告诉我我的 Gitlab 服务器发生了什么事吗？
呵呵，找到答案了
因为pid被拿掉了
那么多git死进程，就像
git 48239 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48254 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48280 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48295 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48322 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48375 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48456 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48457 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48485 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48504 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48521 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48578 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48625 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48666 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48785 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48839 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48931 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48947 36927 0 15:22 ? 00:00:00 [git] <defunct>
触发器是 grack ，当回滚 grack 版本到 1.0.0 时，一切正常。
我们已经使用 Gitlab 作为我们的版本控制系统。
但是独角兽服务今晚崩溃了
with ps -ef | grep unicorn 我看不到任何关于独角兽的进程
当我想使用 nohup bundle exec unicorn -c config/unicorn.rb -E production -p 3000 2>&1 >dev/null &
它说
E, [2015-04-21T05:32:49.215133 #27488] ERROR -- : reaped #<Process::Status: pid 14751 SIGTRAP (signal 5)> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.234264 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.258069 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.274698 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.291256 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.307713 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
[FATAL] Failed to create timer thread: Resource temporarily unavailable
E, [2015-04-21T05:32:49.324021 #27488] ERROR -- : reaped #<Process::Status: pid 14751 exit 1> worker=3
我找到了 unicorn 崩溃时的日志
#
# Fatal error in ../src/platform-linux.cc, line 825
# CHECK_EQ(0, result) failed
# Expected: 0
# Found: 11
#
==== C stack trace ===============================
1: V8_Fatal
2: v8::internal::Thread::Start()
3: v8::internal::Sampler::Start()
4: v8::internal::Isolate::Init(v8::internal::Deserializer*)
5: v8::internal::Snapshot::Initialize(char const*)
6: v8::V8::Initialize()
7: v8::Locker::Initialize(v8::Isolate*)
8: rr::Locker::setupLockAndCall(int*, unsigned long)
9: rr::Locker::doLock(int, unsigned long*, unsigned long)
10: ??
11: ??
12: ??
13: ??
14: ??
15: rb_class_new_instance
16: ??
17: ??
18: ??
19: rb_obj_instance_eval
20: ??
21: ??
22: ??
23: rb_yield
24: rb_ary_each
25: ??
26: ??
27: ??
28: ??
29: ??
30: rb_class_new_instance
31: ??
32: ??
33: ??
34: rb_yield
35: rb_ensure
36: ??
37: ??
38: ??
39: ??
40: ??
41: ??
42: ??
43: ??
44: ??
45: ??
46: ??
47: ??
48: ??
49: ??
50: ??
51: rb_yield
52: rb_ary_each
53: ??
54: ??
55: ??
56: ??
57: ??
58: rb_catch_obj
59: ??
60: ??
61: ??
62: ??
63: rb_yield
64: rb_ary_each
65: ??
66: ??
67: ??
68: rb_iseq_eval
69: ??
70: ??
71: ??
72: ??
73: ??
74: rb_iseq_eval_main
75: ??
76: ruby_exec_node
77: ruby_run_node
78: ??
79: __libc_start_main
80: ??
E, [2015-04-21T04:47:33.135673 #27488] ERROR -- : reaped #<Process::Status: pid 29410 SIGTRAP (signal 5)> worker=3
I, [2015-04-21T04:47:33.154016 #29410] INFO -- : worker=3 ready
error: cannot fork() for git-upload-pack: Cannot allocate memory
X-Accel-Mapping header missing
E, [2015-04-21T04:47:33.135673 #27488] ERROR -- : reaped #<Process::Status: pid 29410 SIGTRAP (signal 5)> worker=3
I, [2015-04-21T04:47:33.154016 #29410] INFO -- : worker=3 ready
error: cannot fork() for git-upload-pack: Cannot allocate memory
X-Accel-Mapping header missing
#
# Fatal error in ../src/platform-linux.cc, line 825
# CHECK_EQ(0, result) failed
# Expected: 0
# Found: 11
当我使用 nohup bundle exec unicorn_rails -l 0.0.0.0:3000 -E production -c config/unicorn.rb 时，一切正常，有人能告诉我我的 Gitlab 服务器发生了什么事吗？
呵呵，找到答案了
因为pid被拿掉了
那么多git死进程，就像
git 48239 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48254 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48280 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48295 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48322 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48375 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48456 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48457 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48485 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48504 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48521 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48578 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48625 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48666 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48785 36927 0 15:22 ? 00:00:00 [git] <defunct>
git 48839 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48931 36927 0 15:29 ? 00:00:00 [git] <defunct>
git 48947 36927 0 15:22 ? 00:00:00 [git] <defunct>
触发器是 grack ，当回滚 grack 版本到 1.0.0 时，一切正常。