web-dev-qa-db-ja.com

Ubuntu 12.04サーバーのサーバーメモリを使い果たしている異常なsendmailアクティビティ

環境

  • ラックスペース
  • Ubuntu 12.04
  • Wordpress
  • MySql

問題

ここ数日、非常に深刻なメモリ不足の問題が発生しています。

問題の原因の1つを解決しました それでもsendmailの非常に疑わしいアクティビティが発生します。

この問題に取り組む方法に関する推奨事項はありますか？それはマルウェアだと思いますが、この種の攻撃を解決した経験はありません。

htop

  1  [|||||||||||||||||||||||||                                                          27.0%]     Tasks: 101, 50 thr; 1 running
  2  [|||||||||||||||||||||||||||||||||||||||||                                          45.7%]     Load average: 12.96 12.55 11.95 
  Mem[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||1183/1995MB]     Uptime: 09:53:28
  Swp[||||                                                                           93/2047MB]

  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
19704 root       20   0  120M 25328  2896 S  2.0  1.2  0:46.16 sendmail: MTA: ./s6HH4rLv009027 gmail.co.: user open
 3298 root       20   0   99M  5612  1684 S  2.0  0.3  2:46.31 sendmail: MTA: s6OABpf4003298 localhost [127.0.0.1]: DATA
 3301 root       20   0   99M  5544  1684 S  2.0  0.3  2:40.89 sendmail: MTA: s6OAGAAh003301 localhost [127.0.0.1]: DATA
19510 root       20   0 26488  2568  1212 R  2.0  0.1  0:23.73 htop
  771 syslog     20   0  244M  3892   516 S  1.0  0.2  2:22.43 rsyslogd -c5
 1226 smmsp      20   0  133M 56328  1396 S  0.0  2.8  1:56.85 sendmail: MSP: ./s6K1OdvJ030780 [127.0.0.1]: client DATA status
32488 root       20   0  102M  7168  2748 S  0.0  0.4  0:00.02 sendmail: MTA: ./s6OAcr6I032488 aspmx.l.google.com.: client EHLO
31723 www-data   39  19  448M 72676 47276 S  0.0  3.6  0:01.14 /usr/sbin/Apache2 -k start
29624 root       20   0  120M 25916  2884 S  0.0  1.3  0:29.65 sendmail: MTA: ./s6NHPdHs002287 todito.com.: user open
  898 mysql      20   0 1315M  105M  3296 S  0.0  5.3 23:25.23 /usr/sbin/mysqld
30966 root       20   0  101M  5092   460 D  0.0  0.2  0:01.52 sendmail: MTA: running queue: /var/spool/mqueue
 5013 mysql      20   0 1315M  105M  3296 S  0.0  5.3  0:25.58 /usr/sbin/mysqld
25504 root       20   0  120M 25904  2900 S  0.0  1.3  0:24.57 sendmail: MTA: ./s6JHcEdS028616 hotamil.com.: user open
 1033 root       20   0  630M  6228  2356 S  0.0  0.3  1:17.85 /usr/local/bin/driveclient --daemon
 1062 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.50 /usr/local/bin/driveclient --daemon
 1082 newrelic   20   0  107M  1576  1072 S  0.0  0.1  0:46.81 /usr/sbin/nrsysmond -c /etc/newrelic/nrsysmond.cfg -p /var/run/nrsysmond.pid
 1089 newrelic   20   0  107M  1576  1072 S  0.0  0.1  0:46.80 /usr/sbin/nrsysmond -c /etc/newrelic/nrsysmond.cfg -p /var/run/nrsysmond.pid
  822 syslog     20   0  244M  3892   516 S  0.0  0.2  1:35.12 rsyslogd -c5
 1061 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.80 /usr/local/bin/driveclient --daemon
 8532 root       20   0  105M  9444   460 D  0.0  0.5  0:06.40 sendmail: MTA: running queue: /var/spool/mqueue
31711 www-data   39  19  445M 75316 52764 S  0.0  3.7  0:01.50 /usr/sbin/Apache2 -k start
27927 root       20   0  120M 25904  2900 S  0.0  1.3  0:32.35 sendmail: MTA: ./s6NKLEhE005721 yahoo.co.: user open
13821 mysql      20   0 1315M  105M  3296 S  0.0  5.3  2:25.39 /usr/sbin/mysqld
31924 mysql      20   0 1315M  105M  3296 S  0.0  5.3  0:49.12 /usr/sbin/mysqld
31713 www-data   39  19  446M 68484 45496 S  0.0  3.4  0:00.79 /usr/sbin/Apache2 -k start
 4195 mysql      20   0 1315M  105M  3296 S  0.0  5.3  0:29.08 /usr/sbin/mysqld
 9799 mysql      20   0 1315M  105M  3296 S  0.0  5.3  2:29.95 /usr/sbin/mysqld
 2664 smmsp      20   0  133M 56424  1476 D  0.0  2.8  1:52.68 sendmail: MSP: ./s6K3MC7s027126 [127.0.0.1]: client DATA status
  853 syslog     20   0  244M  3892   516 S  0.0  0.2  0:47.23 rsyslogd -c5
31714 www-data   39  19  446M 68404 45420 S  0.0  3.3  0:00.73 /usr/sbin/Apache2 -k start
31903 mysql      20   0 1315M  105M  3296 S  0.0  5.3  0:47.96 /usr/sbin/mysqld
 1063 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.40 /usr/local/bin/driveclient --daemon
31600 www-data   39  19  448M 71340 46228 S  0.0  3.5  0:00.92 /usr/sbin/Apache2 -k start
 4308 mysql      20   0 1315M  105M  3296 S  0.0  5.3  0:28.28 /usr/sbin/mysqld
 1064 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.41 /usr/local/bin/driveclient --daemon
31727 www-data   39  19  447M 70324 45756 S  0.0  3.4  0:00.84 /usr/sbin/Apache2 -k start
31725 www-data   39  19  447M 70340 45756 S  0.0  3.4  0:00.86 /usr/sbin/Apache2 -k start
31724 www-data   39  19  447M 70548 45932 S  0.0  3.5  0:00.84 /usr/sbin/Apache2 -k start
 1715 mysql      20   0 1315M  105M  3296 S  0.0  5.3  3:05.00 /usr/sbin/mysqld
23774 root       39  19  425M  6636  4676 S  0.0  0.3  0:06.00 /usr/sbin/Apache2 -k start
 1065 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.35 /usr/local/bin/driveclient --daemon
 1060 root       20   0  630M  6228  2356 S  0.0  0.3  0:12.43 /usr/local/bin/driveclient --daemon
F1Help  F2Setup F3SearchF4FilterF5Tre

巨大な/ var/mail

root@web:/var/mail# ls -alh
total 1.2G
drwxrwsrwt  2 root     mail 4.0K Jul 24 10:51 .
drwxr-xr-x 15 root     root 4.0K Jul 24 00:45 ..
-rw-rw----  1 munin    mail  83K Jul 19 18:48 munin
-rw-------  1 root     mail 1.1G Jul 24 10:51 root
-rw-rw----  1 www-data mail  98M Jul 23 22:34 www-data

私のルートメールアカウントは継続的にメールを送信しています

注：ドメインをfiltered.comに置き換えました

Return-Path: <MAILER-DAEMON>
Received: from localhost (localhost)
    by web.filtered.com (8.14.4/8.14.4/Debian-2ubuntu2) id s6OAqpuv010033;
    Thu, 24 Jul 2014 10:52:51 GMT
Date: Thu, 24 Jul 2014 10:52:51 GMT
From: Mail Delivery Subsystem <MAILER-DAEMON>
Message-Id: <[email protected]>
To: <[email protected]>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
    boundary="s6OAqpuv010033.1406199171/web.filtered.com"
Subject: Returned mail: see transcript for details
Auto-Submitted: auto-generated (failure)

--s6OAqpuw010033.1406199172/web.filtered.com--

From MAILER-DAEMON  Thu Jul 24 10:52:53 2014
Return-Path: <MAILER-DAEMON>
Received: from localhost (localhost)
    by web.filtered.com (8.14.4/8.14.4/Debian-2ubuntu2) id s6OAqq6J010047;
    Thu, 24 Jul 2014 10:52:53 GMT
Date: Thu, 24 Jul 2014 10:52:53 GMT
From: Mail Delivery Subsystem <MAILER-DAEMON>
Message-Id: <[email protected]>
To: postmaster
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
    boundary="s6OAqq6J010047.1406199173/web.filtered.com"
Subject: Postmaster notify: see transcript for details
Auto-Submitted: auto-generated (postmaster-notification)

This is a MIME-encapsulated message

--s6OAqq6J010047.1406199173/web.filtered.com

The original message was received at Thu, 24 Jul 2014 10:52:52 GMT
from localhost
with id s6OAqq6I010047

   ----- The following addresses had permanent fatal errors -----
<[email protected]>
    (reason: 550-5.1.1 The email account that you tried to reach does not exist. Please try)

   ----- Transcript of session follows -----
... while talking to aspmx.l.google.com.:
>>> RCPT To:<[email protected]>
<<< 550-5.1.1 The email account that you tried to reach does not exist. Please try
<<< 550-5.1.1 double-checking the recipient's email address for typos or
<<< 550-5.1.1 unnecessary spaces. Learn more at
<<< 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 sq8si14059110obc.83 - gsmtp
550 5.1.1 <[email protected]>... User unknown
>>> DATA
<<< 503 5.5.1 RCPT first. sq8si14059110obc.83 - gsmtp

--s6OAqq6J010047.1406199173/web.filtered.com
Content-Type: message/delivery-status

Reporting-MTA: dns; web.filtered.com
Received-From-MTA: DNS; localhost
Arrival-Date: Thu, 24 Jul 2014 10:52:52 GMT

Final-Recipient: RFC822; [email protected]
Action: failed
Status: 5.1.1
Remote-MTA: DNS; aspmx.l.google.com
Diagnostic-Code: SMTP; 550-5.1.1 The email account that you tried to reach does not exist. Please try
Last-Attempt-Date: Thu, 24 Jul 2014 10:52:53 GMT

--s6OAqq6J010047.1406199173/web.filtered.com
Content-Type: text/rfc822-headers

Return-Path: <MAILER-DAEMON>
Received: from localhost (localhost)
    by web.filtered.com (8.14.4/8.14.4/Debian-2ubuntu2) id s6OAqq6I010047;
    Thu, 24 Jul 2014 10:52:52 GMT
Date: Thu, 24 Jul 2014 10:52:52 GMT
From: Mail Delivery Subsystem <MAILER-DAEMON>
Message-Id: <[email protected]>
To: <[email protected]>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
    boundary="s6OAqq6I010047.1406199172/web.filtered.com"
Subject: Returned mail: see transcript for details
Auto-Submitted: auto-generated (failure)

--s6OAqq6J010047.1406199173/web.filtered.com--

ps -ef | grep sendmail

root@web:/var/mail# ps -ef | grep sendmail
smmsp     1226     1  0 00:45 ?        00:02:04 sendmail: MSP: ./s6KKDDVU014035 [127.0.0.1]: client DATA status
smmsp     2644  2641  0 01:00 ?        00:00:00 /bin/sh -c test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp
smmsp     2647  2644  0 01:00 ?        00:00:00 /bin/sh /usr/share/sendmail/sendmail cron-msp
smmsp     2664  2647  0 01:00 ?        00:01:58 sendmail: MSP: [127.0.0.1]: idle              
root      3298     1  1 07:57 ?        00:03:16 sendmail: MTA: s6OB1dam003298 localhost [127.0.0.1]: DATA
root      3301     1  1 07:57 ?        00:03:05 sendmail: MTA: server localhost [127.0.0.1] cmd read
root     19675     1  0 11:20 ?        00:00:00 sendmail: MTA: ./s6OBKJuv019675 aspmx.l.google.com.: client DATA 354
root     19689     1  0 11:20 ?        00:00:00 sendmail: MTA: ./s6OBKLuv019689 aspmx.l.google.com.: client DATA 354
root     19800     1  0 11:20 ?        00:00:00 sendmail: MTA: ./s6OBKbuv019800 aspmx.l.google.com.: client DATA 354
root     20178     1  0 11:21 ?        00:00:00 sendmail: MTA: ./s6OBLSuv020178 aspmx.l.google.com.: client DATA 354
root     20270     1  0 11:21 ?        00:00:00 sendmail: MTA: ./s6OBLZuv020270 aspmx.l.google.com.: client DATA 354
root     20537     1  0 11:21 ?        00:00:00 sendmail: MTA: ./s6OBM0uv020537 aspmx.l.google.com.: client DATA 354
root     20646     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBM5uv020646 aspmx.l.google.com.: client DATA 354
root     21006     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMZ6I021006 aspmx.l.google.com.: client DATA 354
root     21015     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMZ6I021015 aspmx.l.google.com.: client DATA 354
root     21027     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMauv021027 aspmx.l.google.com.: client DATA 354
root     21036     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMb6I021036 aspmx.l.google.com.: client DATA 354
root     21063     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMeuv021063 aspmx.l.google.com.: client DATA 354
root     21065     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021065 aspmx.l.google.com.: client DATA 354
root     21086     1  1 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021086 aspmx.l.google.com.: client DATA 354
root     21094     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021094 aspmx.l.google.com.: client DATA 354
root     21098     1  2 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021098 aspmx.l.google.com.: client DATA 354
root     21103     1  1 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021103 aspmx.l.google.com.: client DATA 354
root     21105     1  1 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMguv021105 aspmx.l.google.com.: client DATA 354
root     21108     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OB1dag003298 mx-eu.mail.am0.yahoodns.net.: client MAIL
root     21111     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OBMg6I021111 aspmx.l.google.com.: client RCPT
root     21113     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OAsOi1003301 mx-ha03.web.de.: client greeting
root     21117     1  1 11:22 ?        00:00:00 sendmail: MTA: ./s6OAsOi3003301 gmail-smtp-in.l.google.com.: client DATA status
root     21123     1  0 11:22 ?        00:00:00 sendmail: MTA: ./s6OAsOi5003301 gmail-smtp-in.l.google.com.: client EHLO
root     21127 18604  0 11:22 pts/0    00:00:00 grep --color=auto sendmail

Sendmailステータス

root@web:/var/mail# /etc/init.d/sendmail status
MSP: is run via cron (20m)
MTA: is not running
QUE: Same as MTA

/ var/spool/mqueue

root@web:/var/spool# ls -alh
total 48M
drwxr-xr-x  7 root  root  4.0K Mar 29  2013 .
drwxr-xr-x 15 root  root  4.0K Jul 24 00:45 ..
drwxr-xr-x  5 root  root  4.0K May  1  2012 cron
lrwxrwxrwx  1 root  root     7 May  1  2012 mail -> ../mail
drwxr-s---  2 smmta smmsp  14M Jul 24 11:44 mqueue
drwxrws---  2 smmsp smmsp  34M Jul 24 12:25 mqueue-client
drwxr-xr-x  2 root  root  4.0K Apr 13  2012 plymouth
drwxr-xr-x  2 root  root  4.0K Mar 30  2012 rsyslog


root@web:/var/spool# du -h -d 1
4.0K    ./plymouth
1.6G    ./mqueue    <=====
4.0K    ./rsyslog

root@web:/var/spool/mqueue# more qfs6OBTUZY003298 
V8
T1406201622
K1406201622
N1
P120781
I202/1/476577
MDeferred: 421 4.7.1 : (DYN:T1) http://postmaster.info.aol.com/errors/421dynt1.html
Fbs
$_localhost [127.0.0.1]
$rESMTP
$sweb.anybots.com
${daemon_flags}
${if_addr}127.0.0.1
S<[email protected]>
MDeferred: 421 4.7.1 : (DYN:T1) http://postmaster.info.aol.com/errors/421dynt1.html
rRFC822; [email protected]
RPFD:<[email protected]>
H?P?Return-Path: <?g>
H??Received: from web.anybots.com (localhost [127.0.0.1])
    by web.anybots.com (8.14.4/8.14.4/Debian-2ubuntu2) with ESMTP id s6OBTUZY003298
    for <[email protected]>; Thu, 24 Jul 2014 11:33:42 GMT
H??Received: (from www-data@localhost)
    by web.anybots.com (8.14.4/8.14.4/Submit) id s6JHVJId026134;
    Sat, 19 Jul 2014 17:31:19 GMT
H??Date: Sat, 19 Jul 2014 17:31:19 GMT
H??Message-Id: <[email protected]>
H??X-Authentication-Warning: web.anybots.com: www-data set sender to [email protected] using -f
H??To: [email protected]
H??Subject: Fw:  Hi Generic Drugs Online Products
H??X-PHP-Originating-Script: 33:dirs.php
H??From: "Patty Jennings" <[email protected]>
H??Reply-To:"Patty Jennings" <[email protected]>
H??X-Priority: 3 (Normal)
H??MIME-Version: 1.0
H??Content-Type: text/html; charset="iso-8859-1"
H??Content-Transfer-Encoding: 8bit
.
ubuntu-12.04sendmailspamwordpresslow-memory
5
zabumba

問題は、[〜＃〜] huge [〜＃〜]両方のsendmailキューにある（スパム）メッセージの数が原因である可能性があります。
（ https://serverfault.com/a/490890/163277 を参照）

両方のsendmailキューのメッセージ数を確認する

sendmail -O QueueSortOrder=none -Am -bp
sendmail -O QueueSortOrder=none -Ac -bp

最もメモリを消費するsendmailプロセスは、MTAキュー処理（-Am）のように見えます。残りは、MSAからMTAキューへの転送と、そのような転送後の外部サーバーへの初回配信試行のように見えます。

qtool.plスクリプトを使用して、www-data（Webサーバー）によって送信されたメッセージを別のキュー/ディレクトリに移動できます。これは、sendmail.orgディストリビューションのcontribディレクトリおよびDebian-Linuxによるsendmail-baseパッケージで提供されます。

5
AnFi