我们每8小时左右就会发生崩溃。我们正在Ubuntu 18.04 AWS EC2 Server上将其作为服务运行。错误日志如下:
Jun 3 23:13:12 ip-<IP HERE> <PROJECT NAME>[13506]: -x- GET / 200 4,495ms 13.38kb
Jun 3 23:13:29 ip-<IP HERE> <PROJECT NAME>[13506]: <-- GET /
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: <--- Last few GCs --->
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: [13546:0x2c50070] 16136092 ms: Mark-sweep 1531.3 (1818.9) -> 1531.3 (1787.9) MB, 564.4 / 0.0 ms (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 565 ms) last resort GC in old space requested
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: [13546:0x2c50070] 16136657 ms: Mark-sweep 1531.3 (1787.9) -> 1531.3 (1787.9) MB, 564.6 / 0.0 ms last resort GC in old space requested
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: <--- JS stacktrace --->
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: ==== JS stack trace =========================================
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: Security context: 0x31dffda25891 <JSObject>
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 1: isAlive(aka relayAlive) [/home/ubuntu/<PROJECT NAME>/server/controllers/relay.js:~331] [pc=0x2553280b906c](this=0x86baad75b99 <Object map = 0x1d125244b269>,ip=0x5d343cff359 <String[12]: 10.128.3.231>,port=0x5d343cff381 <String[2]: 80>,id=0x5d343cfede1 <ObjectID map = 0x49650a93921>)
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 2: /* anonymous */(aka /* anonymous */) [/home/ubuntu/<PROJECT NAME>/server/controllers/cpanel.js:190] [bytecode=...
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]:
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 1: node::Abort() [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 2: 0x8cd49c [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 3: v8::Utils::ReportOOMFailure(char const*, bool) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 6: v8::internal::HashTable<v8::internal::SeededNumberDictionary, v8::internal::SeededNumberDictionaryShape>::NewInternal(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 7: v8::internal::HashTable<v8::internal::SeededNumberDictionary, v8::internal::SeededNumberDictionaryShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 8: v8::internal::HashTable<v8::internal::SeededNumberDictionary, v8::internal::SeededNumberDictionaryShape>::EnsureCapacity(v8::internal::Handle<v8::internal::SeededNumberDictionary>, int, v8::internal::PretenureFlag) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 9: v8::internal::Dictionary<v8::internal::SeededNumberDictionary, v8::internal::SeededNumberDictionaryShape>::Add(v8::internal::Handle<v8::internal::SeededNumberDictionary>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, int*) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 10: 0xd80d96 [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 11: v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 12: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 13: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 14: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: 15: 0x255321f042fd
Jun 3 23:13:31 ip-<IP HERE> <PROJECT NAME>[13506]: #033[31m[nodemon] app crashed - waiting for file changes before starting...#033[39m
我已经尝试将以下内容添加为bash别名(使用nodeversion只是为了确保别名能够正常工作:]
alias npm='node --max_old_space_size=8192 /usr/bin/npm'
alias nodeversion='node --version'
我什至尝试了--max-old-space-size标志,使人绝望。以下来自我们的package.json:
"start": "grunt --max-old-space-size=8192",
我正在运行console.log(v8.getHeapStatistics())
以吐出堆,但是heap_size_limit从未超过1.5 GB。我们有足够多的RAM来支持给节点8GB
这是我们运行sudo service <service name> start
时的控制台日志记录:
> [email protected] start /home/ubuntu/<Project Folder>
> grunt --max-old-space-size=8192
Running "concurrent:tasks" (concurrent) task
Running "nodemon:dev" (nodemon) task
Running "watch" task
Waiting...
[nodemon] 1.19.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node app.js`
Mongo Server Running..
KOAN listening on port 17100...
{ total_heap_size: 92770304,
total_heap_size_executable: 3670016,
total_physical_size: 87881664,
total_available_size: 1420343496,
used_heap_size: 60364472,
heap_size_limit: 1501560832,
malloced_memory: 8192,
peak_malloced_memory: 6104104,
does_zap_garbage: 0 }
运行应用程序的服务文件是这个:
Description=<Project Name>
#Requires=After=mysql.service # Requires the mysql service to run first
[Service]
ExecStart=/usr/bin/npm start
# Required on some systems
WorkingDirectory=/home/ubuntu/<Project Folder>
Restart=always
# Restart service after 10 seconds if node service crashes
RestartSec=10
# Output to syslog
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=<Project Name>
User=ubuntu
#Group=<alternate group>
Environment=NODE_ENV=production PORT=17100
[Install]
WantedBy=multi-user.target
该网络应用在崩溃前可以正常运行约4-5个小时。任何帮助将不胜感激。谢谢!
我发现最简单的方法是改用NODE_OPTIONS
环境变量:
"start": "NODE_OPTIONS='--max-old-space-size=8192' grunt"