web-dev-qa-db-ja.com

Hibernate "APPARENT DEADLOCK !!!割り当てられていない保留中のタスクの緊急スレッドを作成しています！"

最近、ログに次のメッセージが表示され、続いてJVMがクラッシュしました（OOMEによる）。私はこれをどうするべきかわからないので、ガイダンスを本当に感謝します。

2015-03-19 21:15:02,457 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.Java [run]:608) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! 
2015-03-19 21:26:29,543 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.Java [run]:624) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@15da1b6b (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@b35b08a (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@51cfdd17 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@19397937
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@5c7d3838
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@7aea62dd
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@55622ff2
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@74004a8
Pool thread stack traces:
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:560)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
        Java.net.SocketOutputStream.socketWrite0(Native Method)
        Java.net.SocketOutputStream.socketWrite(SocketOutputStream.Java:109)
        Java.net.SocketOutputStream.write(SocketOutputStream.Java:153)
        Java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.Java:82)
        Java.io.BufferedOutputStream.flush(BufferedOutputStream.Java:140)
        com.mysql.jdbc.MysqlIO.send(MysqlIO.Java:3227)
        com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.Java:1917)
        com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.Java:2060)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.Java:2536)
        com.mysql.jdbc.ConnectionImpl.configureClientCharacterSet(ConnectionImpl.Java:1751)
        com.mysql.jdbc.ConnectionImpl.initializePropsFromServer(ConnectionImpl.Java:3425)
        com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.Java:2196)
        com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.Java:718)
        com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.Java:46)
        Sun.reflect.GeneratedConstructorAccessor306.newInstance(Unknown Source)
        Sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.Java:45)
        Java.lang.reflect.Constructor.newInstance(Constructor.Java:408)
        com.mysql.jdbc.Util.handleNewInstance(Util.Java:406)
        com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.Java:302)
        com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.Java:282)
        com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.Java:135)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.Java:182)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.Java:171)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.Java:137)
        com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.Java:1014)
        com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.Java:32)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.Java:1810)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:560)


2015-03-19 21:56:59,137 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.Java [run]:608) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! 
2015-03-19 21:56:59,143 [eXistThread-18676] ERROR (XQueryServlet.Java [process]:566) - Java heap space 
Java.lang.OutOfMemoryError: Java heap space
    at org.exist.storage.btree.BTree$BTreeNode.read(BTree.Java:1269)
    at org.exist.storage.btree.BTree$BTreeNode.access$16(BTree.Java:1239)
    at org.exist.storage.btree.BTree.getBTreeNode(BTree.Java:460)
    at org.exist.storage.btree.BTree.scanSequential(BTree.Java:413)
    at org.exist.storage.btree.BTree$BTreeNode.scanNextPage(BTree.Java:2039)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.Java:1835)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.Java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.Java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.Java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.access$12(BTree.Java:1734)
    at org.exist.storage.btree.BTree.query(BTree.Java:379)
    at org.exist.storage.structural.NativeStructuralIndexWorker.scanByType(NativeStructuralIndexWorker.Java:259)
    at org.exist.dom.VirtualNodeSet.getNodesFromIndex(VirtualNodeSet.Java:457)
    at org.exist.dom.VirtualNodeSet.realize(VirtualNodeSet.Java:585)
    at org.exist.dom.VirtualNodeSet.iterator(VirtualNodeSet.Java:860)
    at org.exist.dom.AbstractNodeSet.iterator(AbstractNodeSet.Java:1)
    at org.exist.storage.structural.NativeStructuralIndexWorker.findDescendantsByTagName(NativeStructuralIndexWorker.Java:162)
    at org.exist.xquery.LocationStep.getAttributes(LocationStep.Java:645)
    at org.exist.xquery.LocationStep.eval(LocationStep.Java:434)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.Java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.Java:264)
    at org.exist.xquery.Predicate.selectByNodeSet(Predicate.Java:446)
    at org.exist.xquery.Predicate.evalPredicate(Predicate.Java:326)
    at org.exist.xquery.LocationStep.processPredicate(LocationStep.Java:251)
    at org.exist.xquery.LocationStep.applyPredicate(LocationStep.Java:238)
    at org.exist.xquery.LocationStep.eval(LocationStep.Java:462)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.Java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.Java:264)
    at org.exist.xquery.LetExpr.eval(LetExpr.Java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.Java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.Java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.Java:164)
2015-03-19 21:56:59,147 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.Java [run]:624) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@79180a12 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@243c6d0c (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@50191373 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@3a9d08ca
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@3ecdd11
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@44ff846d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@5ce5850a
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@eec1d04
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@6b8d4d9d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@53e9706d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@23d472cf
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4dbe4f8c
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4c5e0203
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@54ac79fd
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@546e2bad
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@6b13cc83
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@57e185f8
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@60357d68
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@45231180
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@3021aa73
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@6bb437ca
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@2021c9e9
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@7d53637c
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@409c2c97
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@adc5929
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@241ca71a
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@42b26866
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@636b1c33
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@b160466
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4af34669
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@1b53e609
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@2062ebd4
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@1b6cfe8a
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4b7c2380
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4f9be748
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@78108924
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@474b002
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@2ebee32f
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@3e0fe017
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@42aa175b
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@637f5bac
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@3a017b77
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@7b4f2b78
Pool thread stack traces:
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
        Sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.Java:45)
        Java.lang.reflect.Constructor.newInstance(Constructor.Java:408)
        com.mysql.jdbc.Util.handleNewInstance(Util.Java:406)
        com.mysql.jdbc.ResultSetImpl.getInstance(ResultSetImpl.Java:370)
        com.mysql.jdbc.MysqlIO.buildResultSetWithRows(MysqlIO.Java:2532)
        com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.Java:477)
        com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.Java:2510)
        com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.Java:1746)
        com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.Java:2135)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.Java:2536)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.Java:2465)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.Java:1383)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.Java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.Java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.Java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.Java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.Java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.Java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.Java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.Java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.Java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.Java:2596)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.Java:2465)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.Java:1383)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.Java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.Java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.Java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.Java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.Java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.Java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.Java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.Java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.Java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
        Java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.Java:1012)
        Java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.Java:1006)
        com.newrelic.agent.TransactionService.addTransaction(TransactionService.Java:142)
        com.newrelic.agent.Transaction.getTransaction(Transaction.Java:1104)
        com.newrelic.agent.Transaction.getTransaction(Transaction.Java:1087)
        com.newrelic.agent.TracerService$TracerServiceImpl.getTracer(TracerService.Java:136)
        com.newrelic.agent.TracerService.getTracer(TracerService.Java:41)
        com.newrelic.agent.instrumentation.InvocationPoint.invoke(InvocationPoint.Java:55)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.Java)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.Java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.Java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.Java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.Java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.Java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.Java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.Java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.Java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.Java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.Java:547)

次のc3p0構成でhiberanateとc3p0を使用しています。

Hibernateとc3p0には次のMavenアーティファクトを使用しています。

 <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>4.3.6.Final</version>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-c3p0</artifactId>
            <version>4.3.6.Final</version>
        </dependency>

次のc3p0構成で：

  configuration = new Configuration().setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5InnoDBDialect")
  .setProperty("hibernate.connection.provider_class", "org.hibernate.connection.C3P0ConnectionProvider")
  .setProperty("hibernate.c3p0.idle_test_period", "1000")
  .setProperty("hibernate.c3p0.min_size", "20")
  .setProperty("hibernate.c3p0.max_size", "50")
  .setProperty("hibernate.c3p0.timeout", "1800")
  .setProperty("hibernate.c3p0.max_statements", "50")

サーバーの負荷は非常に軽く、1秒あたり最大5回のクエリを実行します。 （Java 8）

javahibernatec3p0
11
Casey Jordan

したがって、直接問題は、接続プールが新しい接続を取得しようとしたが、それらのタスクを取得しようとしたタスクが長時間フリーズしたため、c3p0がタスクをデッドロックして破棄し、スレッドを置き換える必要があると判断したためです。プール。その後、スレッドプールはアイドル接続テストタスクでハングしました。

通常、「ハングした」タスクは、最初の「プールスレッドスタックトレース：」の下にある2番目のスレッドのように見える傾向があります：上記のラベル：実行ネットワークIO完了できません。この2つの状況は奇妙です3つのスレッドのうち、IOでスタックしていません。ほとんど何もし始めていないのに、まだライブではありません。その後、OutOfMemoryErrorが発生し、アイドル接続テストタスクで別のAPPARENT DEADLOCKが発生します。住む。

たぶん、あなたのアプリケーションは、物事が非常に遅くなる原因となっているある種のリソース制限に非常に近いでしょうか？簡単に言うと、このアプリケーションで使用できるメモリの量を増やす（またはメモリフットプリントが小さくなるように変更する）ことができます。スレッドプールがハングしたスレッドを2回目にフラッシュおよび再作成しようとするとOOMEが発生します。これは、直接引き起こされたものではありませんが、スレッドフットプリントの増加が原因である可能性があります。 （以前のログでは、これらの明白なデッドロックがたくさんありますか？JVMスレッドダンプを強制すると、多くのcom.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThreadインスタンスがまだ古いままで、タスクが完了していないのがわかりますか？

いくつかの一般的なコメント：負荷が5を超える同時クエリではない場合、なぜそのような重いプールが必要ですか？なぜmin_size 5-ish、最大10-ishですか？ max_statements設定は、プールで許可する接続数に対して小さすぎます。物事がスムーズに機能するまで、これは完全に省略します。次に、パフォーマンスを少し向上させるために、必要に応じてmore-straightforward-to-reason-about maxStatementsPerConnectionパラメータを設定できます。

ほとんどの場合、アプリケーションのフットプリント（メモリ？スレッド？）を、割り当てられたリソースよりも十分下に保つ必要があります。これは、リソースを増やすか、フットプリントを減らすか、またはリソースフットプリントが限界に達する原因となる可能性のある問題を修正することによって行います。まず、プールを小さくし、使用可能なメモリを大きくして、プールをはるかに小さく構成します。

14
Steve Waldman

私は同じ例外に出くわしました、そしてその理由はdbユーザーの間違ったパスワードでした...

あなたは面白い人です

13
linuxatico

私にとっても認証の問題。データベースサーバー名にドメイン部分を追加したところ、すべてが機能しました。非常に誤解を招くエラー。

1
Gep