バグ #773: 管理画面アクセス時に502 bad gatewayエラー - SDK - Redmine

バグ #773

管理画面アクセス時に502 bad gatewayエラー

山本義治が8年以上前に追加. 約8年前に更新.

ステータス:

進行中

開始日:

2016/05/17

優先度:

急いで

期日:

担当者:

山本義治

進捗 %:

カテゴリ:

ADMIN管理画面開発

作業時間の記録:

対象バージョン:

履歴

#1 山本義治が8年以上前に更新

[root@web1 admin]# pm2 list
┌─────────────┬────┬──────┬───────┬────────┬─────────┬────────┬──────────────┬──────────┐
│ App name    │ id │ mode │ pid   │ status │ restart │ uptime │ memory       │ watching │
├─────────────┼────┼──────┼───────┼────────┼─────────┼────────┼──────────────┼──────────┤
│ www         │ 0  │ fork │ 27292 │ online │ 651     │ 34D    │ 307.961 MB   │ disabled │
│ aggregation │ 1  │ fork │ 27619 │ online │ 88      │ 35D    │ 460.016 MB   │ disabled │
│ sync        │ 2  │ fork │ 13102 │ online │ 6       │ 7D     │ 104.895 MB   │ disabled │
│ postback    │ 3  │ fork │ 31604 │ online │ 5       │ 53D    │ 72.422 MB    │ disabled │
│ wwws        │ 5  │ fork │ 27320 │ online │ 1535    │ 34D    │ 60.645 MB    │ disabled │
│ log_sync    │ 6  │ fork │ 31632 │ online │ 4       │ 53D    │ 358.504 MB   │ disabled │
└─────────────┴────┴──────┴───────┴────────┴─────────┴────────┴──────────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app
[root@web1 admin]# pm2 show wwws
Describing process with id 5 - name wwws
┌───────────────────┬──────────────────────────────────┐
│ status            │ online                           │
│ name              │ wwws                             │
│ id                │ 5                                │
│ path              │ /var/www/x-generation/bin/wwws   │
│ args              │                                  │
│ exec cwd          │ /var/www/x-generation            │
│ error log path    │ /root/.pm2/logs/wwws-error-5.log │
│ out log path      │ /root/.pm2/logs/wwws-out-5.log   │
│ pid path          │ /root/.pm2/pids/wwws-5.pid       │
│ mode              │ fork_mode                        │
│ node v8 arguments │                                  │
│ watch & reload    │ ✘                                │
│ interpreter       │ node                             │
│ restarts          │ 1535                             │
│ unstable restarts │ 0                                │
│ uptime            │ 34D                              │
│ created at        │ 2016-04-12T11:39:25.516Z         │
└───────────────────┴──────────────────────────────────┘
Revision control metadata
┌──────────────────┬────────────────────────────────────────────────────┐
│ revision control │ git                                                │
│ remote url       │ ssh://admin@49.212.24.211/opt/git/x-generation.git │
│ repository root  │ /var/www/x-generation                              │
│ last update      │ 2016-04-12T11:39:10.703Z                           │
│ revision         │ 3ec9b4168486dc40d25cd90021689b1e2097e978           │
│ comment          │ Merge branch 'release/1.0.27'                      │
│ branch           │ master                                             │
└──────────────────┴────────────────────────────────────────────────────┘
Probes value
┌────────────┬────────┐
│ Loop delay │ 45.3ms │
└────────────┴────────┘
[root@web1 admin]# tail -f /root/.pm2/logs/wwws-out-5.log
GET /plugins/iCheck/square/grey.css 304 5305.646 ms - -
GET /stylesheets/skins/skin-black.min.css 304 4247.830 ms - -
GET /stylesheets/style.css 304 3832.175 ms - -
GET /plugins/jQuery/jQuery-2.1.3.min.js 304 4484.036 ms - -
GET /plugins/bootstrap/bootstrap.min.js 304 4256.746 ms - -
GET /plugins/iCheck/icheck.min.js 304 3967.854 ms - -
GET /plugins/iCheck/square/grey.png 304 2914.597 ms - -
GET /plugins/font-awesome/fonts/fontawesome-webfont.woff2?v=4.3.0 200 1376.027 ms - 56780
loginCheck:2016-05-17 09:39:12
GET / - - ms - -
[root@web1 admin]# tail -f /root/.pm2/logs/wwws-error-5.log 
Mongoose: appdailylogs.find({ date: new Date("Sun, 15 May 2016 15:00:00 GMT"), app: ObjectId("56494986fd096ced117bb6c4") }) { sort: { date: -1 }, limit: 1, fields: {} }  
Mongoose: appdailylogs.find({ date: new Date("Mon, 16 May 2016 15:00:00 GMT"), app: ObjectId("56494986fd096ced117bb6c4") }) { sort: { date: -1 }, limit: 1, fields: {} }  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 5629f3ecbc770da91f31aab7, date: { '$gte': Tue May 10 2016 00:00:00 GMT+0900 (JST), '$lte': Mon May 16 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$group': { _id: null, click: { '$avg': '$click' }, install: { '$avg': '$install' }, conversion: { '$avg': '$conversion' }, revisit: { '$avg': '$revisit' }, event: { '$avg': '$event' }, retention: { '$avg': '$retention' }, uu: { '$avg': '$uu' }, pu: { '$avg': '$pu' }, sale: { '$avg': '$sale' }, promotion: { '$avg': '$promotion' }, arpu: { '$avg': '$arpu' }, arppu: { '$avg': '$arppu' }, cpi: { '$avg': '$cpi' }, cpa: { '$avg': '$cpa' }, purchase_rate: { '$avg': '$purchase_rate' }, roas: { '$avg': '$roas' }, ranking: { '$avg': '$ranking' } } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 5629f3ecbc770da91f31aab7, date: { '$gte': Tue May 17 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$sort': { dateTime: 1 } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 5629f499bc770da91f31aab9, date: { '$gte': Tue May 10 2016 00:00:00 GMT+0900 (JST), '$lte': Mon May 16 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$group': { _id: null, click: { '$avg': '$click' }, install: { '$avg': '$install' }, conversion: { '$avg': '$conversion' }, revisit: { '$avg': '$revisit' }, event: { '$avg': '$event' }, retention: { '$avg': '$retention' }, uu: { '$avg': '$uu' }, pu: { '$avg': '$pu' }, sale: { '$avg': '$sale' }, promotion: { '$avg': '$promotion' }, arpu: { '$avg': '$arpu' }, arppu: { '$avg': '$arppu' }, cpi: { '$avg': '$cpi' }, cpa: { '$avg': '$cpa' }, purchase_rate: { '$avg': '$purchase_rate' }, roas: { '$avg': '$roas' }, ranking: { '$avg': '$ranking' } } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 5629f499bc770da91f31aab9, date: { '$gte': Tue May 17 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$sort': { dateTime: 1 } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 56493e9efd096ced117bb6bd, date: { '$gte': Tue May 10 2016 00:00:00 GMT+0900 (JST), '$lte': Mon May 16 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$group': { _id: null, click: { '$avg': '$click' }, install: { '$avg': '$install' }, conversion: { '$avg': '$conversion' }, revisit: { '$avg': '$revisit' }, event: { '$avg': '$event' }, retention: { '$avg': '$retention' }, uu: { '$avg': '$uu' }, pu: { '$avg': '$pu' }, sale: { '$avg': '$sale' }, promotion: { '$avg': '$promotion' }, arpu: { '$avg': '$arpu' }, arppu: { '$avg': '$arppu' }, cpi: { '$avg': '$cpi' }, cpa: { '$avg': '$cpa' }, purchase_rate: { '$avg': '$purchase_rate' }, roas: { '$avg': '$roas' }, ranking: { '$avg': '$ranking' } } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 56493e9efd096ced117bb6bd, date: { '$gte': Tue May 17 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$sort': { dateTime: 1 } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 56494986fd096ced117bb6c4, date: { '$gte': Tue May 10 2016 00:00:00 GMT+0900 (JST), '$lte': Mon May 16 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$group': { _id: null, click: { '$avg': '$click' }, install: { '$avg': '$install' }, conversion: { '$avg': '$conversion' }, revisit: { '$avg': '$revisit' }, event: { '$avg': '$event' }, retention: { '$avg': '$retention' }, uu: { '$avg': '$uu' }, pu: { '$avg': '$pu' }, sale: { '$avg': '$sale' }, promotion: { '$avg': '$promotion' }, arpu: { '$avg': '$arpu' }, arppu: { '$avg': '$arppu' }, cpi: { '$avg': '$cpi' }, cpa: { '$avg': '$cpa' }, purchase_rate: { '$avg': '$purchase_rate' }, roas: { '$avg': '$roas' }, ranking: { '$avg': '$ranking' } } } ]) {}  
Mongoose: appdailylogs.aggregate([ { '$match': { app: 56494986fd096ced117bb6c4, date: { '$gte': Tue May 17 2016 00:00:00 GMT+0900 (JST) } } }, { '$project': { dateTime: { '$dateToString': { format: '%Y-%m-%d', date: { '$add': [ '$date', 32400000 ] } } }, click: true, install: true, conversion: true, revisit: true, event: true, retention: { '$divide': [ { '$subtract': [ { '$multiply': [ '$retention', 100 ] }, { '$mod': [ { '$multiply': [ '$retention', 100 ] }, 1 ] } ] }, 100 ] }, uu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$uu', 100 ] }, { '$mod': [ { '$multiply': [ '$uu', 100 ] }, 1 ] } ] }, 100 ] }, pu: { '$divide': [ { '$subtract': [ { '$multiply': [ '$pu', 100 ] }, { '$mod': [ { '$multiply': [ '$pu', 100 ] }, 1 ] } ] }, 100 ] }, sale: true, promotion: true, ranking: { '$divide': [ { '$subtract': [ { '$multiply': [ '$ranking', 100 ] }, { '$mod': [ { '$multiply': [ '$ranking', 100 ] }, 1 ] } ] }, 100 ] }, arpu: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, arppu: { '$cond': [ { '$eq': [ '$pu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpi: { '$cond': [ { '$eq': [ '$install', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, cpa: { '$cond': [ { '$eq': [ '$conversion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, purchase_rate: { '$cond': [ { '$eq': [ '$uu', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] }, roas: { '$cond': [ { '$eq': [ '$promotion', 0 ] }, null, { '$divide': [ { '$subtract': [ { '$multiply': [ [Object], 100 ] }, { '$mod': [ [Object], 1 ] } ] }, 100 ] } ] } } }, { '$sort': { dateTime: 1 } } ]) {}  
[root@web1 admin]# tail -f /var/log/nginx/access.log
153.156.40.213 - - [17/May/2016:09:43:39 +0900] "GET / HTTP/1.1" 502 172 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-" 
180.76.15.31 - - [17/May/2016:09:51:03 +0900] "GET / HTTP/1.1" 302 53 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-" 
180.76.15.150 - - [17/May/2016:09:51:04 +0900] "GET / HTTP/1.1" 302 53 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-" 
36.13.69.73 - - [17/May/2016:10:36:38 +0900] "GET /admin/ HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:37:41 +0900] "GET /admin/ HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:39:47 +0900] "GET /admin/ HTTP/1.1" 504 537 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:41:47 +0900] "GET /favicon.ico HTTP/1.1" 504 537 "http://i-joji.com/admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:43:35 +0900] "GET /admin/ HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:43:48 +0900] "GET /admin/ HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
36.13.69.73 - - [17/May/2016:10:44:12 +0900] "GET /admin/ HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 
[root@web1 admin]# tail -f /var/log/nginx/error.log
2016/05/17 05:52:22 [error] 23267#0: *386421 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.6, server: i-joji.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "i-joji.com" 
2016/05/17 05:52:44 [error] 23267#0: *386424 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.156, server: i-joji.com, request: "GET / HTTP/1.1", upstream: "http://[::1]:3000/", host: "i-joji.com" 
2016/05/17 05:53:44 [error] 23267#0: *386424 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.156, server: i-joji.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "i-joji.com" 
2016/05/17 09:39:57 [error] 23267#0: *386475 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 153.156.40.213, server: i-joji.com, request: "GET / HTTP/1.1", upstream: "https://127.0.0.1:3001/", host: "i-joji.com" 
2016/05/17 09:50:20 [error] 23267#0: *386480 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.31, server: i-joji.com, request: "GET / HTTP/1.1", upstream: "http://[::1]:3000/", host: "i-joji.com" 
2016/05/17 10:37:38 [error] 23267#0: *386487 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 36.13.69.73, server: i-joji.com, request: "GET /admin/ HTTP/1.1", upstream: "http://[::1]:3000/admin/", host: "i-joji.com" 
2016/05/17 10:38:46 [error] 23267#0: *386490 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 36.13.69.73, server: i-joji.com, request: "GET /admin/ HTTP/1.1", upstream: "http://127.0.0.1:3000/admin/", host: "i-joji.com" 
2016/05/17 10:39:47 [error] 23267#0: *386490 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 36.13.69.73, server: i-joji.com, request: "GET /admin/ HTTP/1.1", upstream: "http://[::1]:3000/admin/", host: "i-joji.com" 
2016/05/17 10:40:47 [error] 23267#0: *386490 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 36.13.69.73, server: i-joji.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:3000/favicon.ico", host: "i-joji.com", referrer: "http://i-joji.com/admin/" 
2016/05/17 10:41:47 [error] 23267#0: *386490 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 36.13.69.73, server: i-joji.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://[::1]:3000/favicon.ico", host: "i-joji.com", referrer: "http://i-joji.com/admin/" 
[root@web1 admin]# pm2 restart wwws
[PM2] restartProcessId process id 5
┌─────────────┬────┬──────┬───────┬────────┬─────────┬────────┬──────────────┬──────────┐
│ App name    │ id │ mode │ pid   │ status │ restart │ uptime │ memory       │ watching │
├─────────────┼────┼──────┼───────┼────────┼─────────┼────────┼──────────────┼──────────┤
│ www         │ 0  │ fork │ 27292 │ online │ 651     │ 34D    │ 362.391 MB   │ disabled │
│ aggregation │ 1  │ fork │ 27619 │ online │ 88      │ 35D    │ 460.109 MB   │ disabled │
│ sync        │ 2  │ fork │ 13102 │ online │ 6       │ 7D     │ 91.020 MB    │ disabled │
│ postback    │ 3  │ fork │ 31604 │ online │ 5       │ 53D    │ 156.527 MB   │ disabled │
│ wwws        │ 5  │ fork │ 28858 │ online │ 1536    │ 0s     │ 14.180 MB    │ disabled │
│ log_sync    │ 6  │ fork │ 31632 │ online │ 4       │ 53D    │ 211.441 MB   │ disabled │
└─────────────┴────┴──────┴───────┴────────┴─────────┴────────┴──────────────┴──────────┘

nodejs再起動後、正常にアクセスできること確認

#2 山本義治が8年以上前に更新

[root@web1 admin]# tail -f /var/log/nginx/error.log
2016/05/17 11:18:47 [warn] 23267#0: *387790 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/3/56/0000002563 while reading upstream, client: 178.140.232.252, server: i-joji.com, request: "GET /plugins/bootstrap/bootstrap.min.css HTTP/1.1", upstream: "https://[::1]:3001/plugins/bootstrap/bootstrap.min.css", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:18:51 [warn] 23267#0: *387803 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/4/56/0000002564 while reading upstream, client: 37.204.159.37, server: i-joji.com, request: "GET /plugins/font-awesome/fonts/fontawesome-webfont.eot? HTTP/1.1", upstream: "https://[::1]:3001/plugins/font-awesome/fonts/fontawesome-webfont.eot", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:18:52 [warn] 23267#0: *387795 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/5/56/0000002565 while reading upstream, client: 128.68.51.76, server: i-joji.com, request: "GET /stylesheets/app.css HTTP/1.1", upstream: "https://127.0.0.1:3001/stylesheets/app.css", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:18:55 [warn] 23267#0: *387815 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/6/56/0000002566 while reading upstream, client: 128.68.100.252, server: i-joji.com, request: "GET /plugins/font-awesome/fonts/fontawesome-webfont.eot? HTTP/1.1", upstream: "https://[::1]:3001/plugins/font-awesome/fonts/fontawesome-webfont.eot", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:18:58 [warn] 23267#0: *387817 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/7/56/0000002567 while reading upstream, client: 95.25.48.48, server: i-joji.com, request: "GET /plugins/jQuery/jQuery-2.1.3.min.js HTTP/1.1", upstream: "https://127.0.0.1:3001/plugins/jQuery/jQuery-2.1.3.min.js", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:19:02 [warn] 23267#0: *387822 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/8/56/0000002568 while reading upstream, client: 46.251.93.253, server: i-joji.com, request: "GET /plugins/bootstrap/bootstrap.min.css HTTP/1.1", upstream: "https://[::1]:3001/plugins/bootstrap/bootstrap.min.css", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:19:11 [warn] 23267#0: *387828 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/9/56/0000002569 while reading upstream, client: 176.77.64.245, server: i-joji.com, request: "GET /stylesheets/app.css HTTP/1.1", upstream: "https://[::1]:3001/stylesheets/app.css", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:19:20 [warn] 23267#0: *387832 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/0/57/0000002570 while reading upstream, client: 95.24.35.201, server: i-joji.com, request: "GET /plugins/jQuery/jQuery-2.1.3.min.js HTTP/1.1", upstream: "https://[::1]:3001/plugins/jQuery/jQuery-2.1.3.min.js", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:19:23 [warn] 23267#0: *387834 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/1/57/0000002571 while reading upstream, client: 94.180.141.189, server: i-joji.com, request: "GET /plugins/bootstrap/bootstrap.min.css HTTP/1.1", upstream: "https://127.0.0.1:3001/plugins/bootstrap/bootstrap.min.css", host: "i-joji.com", referrer: "https://i-joji.com/login" 
2016/05/17 11:19:32 [warn] 23267#0: *387842 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/2/57/0000002572 while reading upstream, client: 109.173.20.42, server: i-joji.com, request: "GET /stylesheets/app.css HTTP/1.1", upstream: "https://127.0.0.1:3001/stylesheets/app.css", host: "i-joji.com", referrer: "https://i-joji.com/login"

キャッシュファイルのバッファサイズが容量オーバーして、メモリからファイルへ保存している

#3 山本義治が8年以上前に更新

top - 12:30:24 up 206 days, 20:28,  2 users,  load average: 2.28, 2.52, 2.78
Tasks: 133 total,   1 running, 132 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.3%sy,  0.0%ni, 66.3%id, 33.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1922072k total,  1851656k used,    70416k free,     1200k buffers
Swap:  2097148k total,  1696340k used,   400808k free,    14120k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                       
27619 root      20   0 1529m 468m 2868 S  0.0 24.9 729:28.40 node /var/www/x-generation/cron/aggregation.js                                                                                                 
31632 root      20   0 1296m 193m 2380 D  1.0 10.3  72:51.11 node /var/www/x-generation/cron/log_sync.js                                                                                                    
31604 root      20   0 1290m 347m 2384 S  0.0 18.5  72:43.63 node /var/www/x-generation/cron/postback.js                                                                                                    
27292 root      20   0 1242m 220m 2692 S  0.0 11.8  46:41.52 node /var/www/x-generation/bin/www                                                                                                             
28858 root      20   0 1125m  76m 4372 S  0.0  4.1   0:18.91 node /var/www/x-generation/bin/wwws                                                                                                            
13102 root      20   0  810m  98m 3052 S  0.0  5.3  24:14.20 node /var/www/x-generation/cron/sync.js

iowaitが発生している
swapを使いきってディスクI/Oが発生している模様

#4 山本義治が8年以上前に更新

root@web1 admin]# pm2 stop aggregation
[PM2] Stopping aggregation
[PM2] stopProcessId process id 1
┌─────────────┬────┬──────┬───────┬─────────┬─────────┬────────┬──────────────┬──────────┐
│ App name    │ id │ mode │ pid   │ status  │ restart │ uptime │ memory       │ watching │
├─────────────┼────┼──────┼───────┼─────────┼─────────┼────────┼──────────────┼──────────┤
│ www         │ 0  │ fork │ 27292 │ online  │ 651     │ 34D    │ 200.523 MB   │ disabled │
│ aggregation │ 1  │ fork │ 0     │ stopped │ 88      │ 0      │ 0 B          │ disabled │
│ sync        │ 2  │ fork │ 13102 │ online  │ 6       │ 7D     │ 104.941 MB   │ disabled │
│ postback    │ 3  │ fork │ 31604 │ online  │ 5       │ 53D    │ 337.219 MB   │ disabled │
│ wwws        │ 5  │ fork │ 28858 │ online  │ 1536    │ 99m    │ 75.691 MB    │ disabled │
│ log_sync    │ 6  │ fork │ 31632 │ online  │ 4       │ 53D    │ 207.906 MB   │ disabled │
└─────────────┴────┴──────┴───────┴─────────┴─────────┴────────┴──────────────┴──────────┘

集計処理プロセスを停止してみる

top - 12:34:06 up 206 days, 20:32,  2 users,  load average: 1.89, 2.59, 2.77
Tasks: 133 total,   1 running, 132 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.3%sy,  0.0%ni, 66.6%id, 32.8%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1922072k total,  1474968k used,   447104k free,     4972k buffers
Swap:  2097148k total,  1251204k used,   845944k free,    25808k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                       
31632 root      20   0 1296m 251m 2380 D  1.0 13.4  72:51.80 node /var/www/x-generation/cron/log_sync.js                                                                                                    
31604 root      20   0 1290m 338m 2384 S  0.0 18.0  72:43.78 node /var/www/x-generation/cron/postback.js                                                                                                    
27292 root      20   0 1242m 201m 2684 S  0.0 10.7  46:41.67 node /var/www/x-generation/bin/www                                                                                                             
28858 root      20   0 1125m  76m 4220 S  0.0  4.1   0:19.08 node /var/www/x-generation/bin/wwws                                                                                                            
13102 root      20   0  810m 114m 3104 S  0.0  6.1  24:15.02 node /var/www/x-generation/cron/sync.js

メモリ使用量は減っているがiowaitは変わらない

[root@web1 admin]# pm2 stop www
[PM2] Stopping www
[PM2] stopProcessId process id 0
┌─────────────┬────┬──────┬───────┬─────────┬─────────┬────────┬──────────────┬──────────┐
│ App name    │ id │ mode │ pid   │ status  │ restart │ uptime │ memory       │ watching │
├─────────────┼────┼──────┼───────┼─────────┼─────────┼────────┼──────────────┼──────────┤
│ www         │ 0  │ fork │ 0     │ stopped │ 651     │ 0      │ 0 B          │ disabled │
│ aggregation │ 1  │ fork │ 0     │ stopped │ 88      │ 0      │ 0 B          │ disabled │
│ sync        │ 2  │ fork │ 13102 │ online  │ 6       │ 7D     │ 114.508 MB   │ disabled │
│ postback    │ 3  │ fork │ 31604 │ online  │ 5       │ 53D    │ 339.277 MB   │ disabled │
│ wwws        │ 5  │ fork │ 28858 │ online  │ 1536    │ 104m   │ 78.918 MB    │ disabled │
│ log_sync    │ 6  │ fork │ 31632 │ online  │ 4       │ 53D    │ 363.914 MB   │ disabled │
└─────────────┴────┴──────┴───────┴─────────┴─────────┴────────┴──────────────┴──────────┘

wwwプロセス停止

top - 12:41:47 up 206 days, 20:40,  2 users,  load average: 0.25, 0.95, 1.91
Tasks: 131 total,   1 running, 130 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.0%sy,  0.0%ni, 98.3%id,  1.3%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1922072k total,  1482436k used,   439636k free,     6448k buffers
Swap:  2097148k total,   719940k used,  1377208k free,    32396k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                       
31632 root      20   0 1296m 366m 2380 S  0.0 19.5  72:52.98 node /var/www/x-generation/cron/log_sync.js                                                                                                    
31604 root      20   0 1290m 341m 2384 S  0.0 18.2  72:44.10 node /var/www/x-generation/cron/postback.js                                                                                                    
28858 root      20   0 1137m 151m 5868 S  0.0  8.1   0:22.18 node /var/www/x-generation/bin/wwws                                                                                                            
13102 root      20   0  810m 117m 3108 S  0.0  6.2  24:15.83 node /var/www/x-generation/cron/sync.js

[root@web1 admin]# iostat -dx 3
Linux 2.6.32-573.7.1.el6.x86_64 (web1.i-joji.com)     2016年05月17日     _x86_64_    (3 CPU)

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda              31.10     4.07    7.50    8.34   313.01    99.30    26.03     0.16   10.19   4.22   6.68

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               1.00     0.33    1.67    0.67    21.33     8.00    12.57     0.02   10.29   6.43   1.50

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda              41.67     4.00   47.67    1.33   714.67    42.67    15.46     0.34    6.67   6.84  33.53

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda              31.67     0.33   32.67    0.67   514.67     8.00    15.68     0.23    7.41   6.81  22.70

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               0.67     2.00    1.67    1.33    18.67    26.67    15.11     0.02    7.00   7.00   2.10

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               1.66     0.00    1.99    0.33    29.24     2.66    13.71     0.04   15.86  10.43   2.43

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               3.33     3.00    1.00    1.00    34.67    32.00    33.33     0.01    4.17   4.00   0.80

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               4.00     0.67    3.33    0.67    58.67    10.67    17.33     0.04   11.00   6.75   2.70

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
vda               3.00     3.67    2.00    3.33    40.00    56.00    18.00     0.02    3.06   2.63   1.40

iowaitが一気に下がった

[root@web1 admin]# cd /etc/nginx/conf.d/
[root@web1 conf.d]# ll
合計 20
-rw-r--r-- 1 root root 1097  4月 22 00:34 2015 default.conf.bkup
-rw-r--r-- 1 root root  427  4月 22 00:34 2015 example_ssl.conf
-rw-r--r-- 1 root root  491 10月 23 15:46 2015 node-app.conf
drwxr-xr-x 2 root root 4096 10月 23 19:18 2015 ssl
-rw-r--r-- 1 root root  797 11月 18 01:48 2015 ssl.conf
[root@web1 conf.d]# vi node-app.conf 
[root@web1 conf.d]# mv node-app.conf node-app.conf.bkup
[root@web1 conf.d]# mv example_ssl.conf example_ssl.conf.bkup
[root@web1 conf.d]# ll
合計 20
-rw-r--r-- 1 root root 1097  4月 22 00:34 2015 default.conf.bkup
-rw-r--r-- 1 root root  427  4月 22 00:34 2015 example_ssl.conf.bkup
-rw-r--r-- 1 root root  491 10月 23 15:46 2015 node-app.conf.bkup
drwxr-xr-x 2 root root 4096 10月 23 19:18 2015 ssl
-rw-r--r-- 1 root root  797 11月 18 01:48 2015 ssl.conf
[root@web1 conf.d]# chkconfig 
abrt-ccpp          0:off    1:off    2:off    3:off    4:off    5:off    6:off
abrt-oops          0:off    1:off    2:off    3:off    4:off    5:off    6:off
abrtd              0:off    1:off    2:off    3:off    4:off    5:off    6:off
acpid              0:off    1:off    2:on    3:on    4:on    5:on    6:off
atd                0:off    1:off    2:off    3:on    4:on    5:on    6:off
auditd             0:off    1:off    2:off    3:off    4:off    5:off    6:off
blk-availability    0:off    1:on    2:on    3:on    4:on    5:on    6:off
cpuspeed           0:off    1:on    2:off    3:off    4:off    5:off    6:off
crond              0:off    1:off    2:on    3:on    4:on    5:on    6:off
datadog-agent      0:off    1:off    2:on    3:on    4:on    5:on    6:off
dovecot            0:off    1:off    2:on    3:on    4:on    5:on    6:off
haldaemon          0:off    1:off    2:off    3:off    4:off    5:off    6:off
htcacheclean       0:off    1:off    2:off    3:off    4:off    5:off    6:off
httpd              0:off    1:off    2:off    3:off    4:off    5:off    6:off
ip6tables          0:off    1:off    2:on    3:on    4:on    5:on    6:off
iptables           0:off    1:off    2:on    3:on    4:on    5:on    6:off
irqbalance         0:off    1:off    2:off    3:on    4:on    5:on    6:off
kdump              0:off    1:off    2:off    3:off    4:off    5:off    6:off
lvm2-monitor       0:off    1:on    2:off    3:off    4:off    5:off    6:off
mdmonitor          0:off    1:off    2:off    3:off    4:off    5:off    6:off
messagebus         0:off    1:off    2:off    3:off    4:off    5:off    6:off
netconsole         0:off    1:off    2:off    3:off    4:off    5:off    6:off
netfs              0:off    1:off    2:off    3:off    4:off    5:off    6:off
network            0:off    1:off    2:on    3:on    4:on    5:on    6:off
nginx              0:off    1:off    2:on    3:on    4:on    5:on    6:off
ntpd               0:off    1:off    2:on    3:on    4:on    5:on    6:off
ntpdate            0:off    1:off    2:on    3:on    4:on    5:on    6:off
portreserve        0:off    1:off    2:on    3:on    4:on    5:on    6:off
postfix            0:off    1:off    2:on    3:on    4:on    5:on    6:off
psacct             0:off    1:off    2:off    3:off    4:off    5:off    6:off
quota_nld          0:off    1:off    2:off    3:off    4:off    5:off    6:off
rdisc              0:off    1:off    2:off    3:off    4:off    5:off    6:off
restorecond        0:off    1:off    2:off    3:off    4:off    5:off    6:off
rngd               0:off    1:off    2:off    3:off    4:off    5:off    6:off
rsyslog            0:off    1:off    2:on    3:on    4:on    5:on    6:off
saslauthd          0:off    1:off    2:on    3:on    4:on    5:on    6:off
smartd             0:off    1:off    2:off    3:off    4:off    5:off    6:off
sshd               0:off    1:off    2:on    3:on    4:on    5:on    6:off
svnserve           0:off    1:off    2:off    3:off    4:off    5:off    6:off
sysstat            0:off    1:on    2:on    3:on    4:on    5:on    6:off
udev-post          0:off    1:on    2:off    3:off    4:off    5:off    6:off
yum-updateonboot    0:off    1:off    2:off    3:off    4:off    5:off    6:off
[root@web1 conf.d]# /etc/init.d/nginx restart
nginx を停止中:                                            [  OK  ]
nginx を起動中:                                            [  OK  ]

www:3000ポートが起動していたので停止

#5 山本義治が8年以上前に更新

ステータス を新規から解決に変更

#6 山本義治が約8年前に更新

ステータス を解決から 進行中 に変更

メモリ使用量モニタリング¶

[root@web1 admin]# pm2 monit
⌬ PM2 monitoring (To go further check out https://app.keymetrics.io)

● www                                 [                              ] stopped
[0] [fork_mode]                        [                              ] 0 B

● aggregation                         [                              ] 0 %
[1] [fork_mode]                        [||||||||||||||                ] 175.480 MB

● sync                                [                              ] 0 %
[2] [fork_mode]                        [|||||||||                     ] 120.535 MB

● postback                            [                              ] 0 %
[3] [fork_mode]                        [||||||||||||                  ] 156.965 MB

● wwws                                [                              ] 0 %
[5] [fork_mode]                        [|||||||                       ] 88.160 MB

● log_sync                            [                              ] 0 %
[6] [fork_mode]                        [|||||||||                     ] 585.215 MB

pm2メモリ閾値超えたら再起動¶

http://pm2.keymetrics.io/docs/usage/monitoring/#max-memory-restart

[root@web1 admin]# pm2 start aggregation --max-memory-restart 200M
[root@web1 admin]# pm2 start sync --max-memory-restart 200M
[root@web1 admin]# pm2 start postback --max-memory-restart 200M
[root@web1 admin]# pm2 start wwws --max-memory-restart 200M
[root@web1 admin]# pm2 start log_sync --max-memory-restart 200M

[root@web1 admin]# pm2 monit
⌬ PM2 monitoring (To go further check out https://app.keymetrics.io)

● www                                 [                              ] stopped
[0] [fork_mode]                        [                              ] 0 B

● aggregation                         [                              ] 2 %
[1] [fork_mode]                        [||||||                        ] 80.191 MB

● sync                                [                              ] 0 %
[2] [fork_mode]                        [|||||                         ] 74.426 MB

● postback                            [                              ] 0 %
[3] [fork_mode]                        [|||||||                       ] 94.902 MB

● wwws                                [                              ] 0 %
[5] [fork_mode]                        [||||||                        ] 86.684 MB

● log_sync                            [                              ] 0 %
[6] [fork_mode]                        [|||||||                       ] 95.074 MB

他の形式にエクスポート: Atom PDF

SDK

チケット

カスタムクエリ

バグ #773

管理画面アクセス時に502 bad gatewayエラー

履歴

#1 山本義治が8年以上前に更新

#2 山本義治が8年以上前に更新

#3 山本義治が8年以上前に更新

#4 山本義治が8年以上前に更新

#5 山本義治が8年以上前に更新

#6 山本義治が約8年前に更新

メモリ使用量モニタリング¶

pm2メモリ閾値超えたら再起動¶

SDK

チケット

カスタムクエリ

バグ #773

管理画面アクセス時に502 bad gatewayエラー

履歴

#1 山本 義治 が8年以上前に更新

#2 山本 義治 が8年以上前に更新

#3 山本 義治 が8年以上前に更新

#4 山本 義治 が8年以上前に更新

#5 山本 義治 が8年以上前に更新

#6 山本 義治 が約8年前に更新

メモリ使用量モニタリング¶

pm2メモリ閾値超えたら再起動¶

#1 山本義治が8年以上前に更新

#2 山本義治が8年以上前に更新

#3 山本義治が8年以上前に更新

#4 山本義治が8年以上前に更新

#5 山本義治が8年以上前に更新

#6 山本義治が約8年前に更新