使用ORMLite时出现死锁

Question

我有一个多线程服务器Java应用程序，该应用程序接收请求并通过OrmLite对Postgres DB进行查询/更新。在负载下，有几个对同一数据库行感兴趣的请求。 Thread1可能会更改select，然后再更改update。同时Thread2尝试类似的操作。当前未同步，并且未在事务内部完成。毫不奇怪，Thread1可能看不到Thread2的更新。没关系（Thread2可以覆盖Thread1的结果），这不是我的问题。

但是，当运行该应用程序一段时间后，我陷入了僵局，这导致所有可用的数据库连接都用完（然后崩溃）。看来这不是一个标准的死锁（具有循环锁依赖性），而是大多数线程正在等待锁，并且拥有此锁的线程似乎正在等待套接字读取（可能不会发生，请参阅下文）。

使用中

• OrmLite 5.1，
• JVM是Java 1.8.0_251热点客户端VM，
• Postgres JDBC 42.2.9

我应该如何解决这个问题？

下面是线程转储的相关部分（由https://spotify.github.io/threaddump-analyzer分析）

持有主锁（0x00000000c0179e18）的线程似乎正在套接字上等待：

"RaspService-2089": running, holding [0x00000000c0179e18, 0x00000000c2c1f6c0]
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
    at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
    at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
    at org.postgresql.core.PGStream.receiveChar(PGStream.java:335)
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:505)
    at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:141)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:211)
    at org.postgresql.Driver.makeConnection(Driver.java:458)
    at org.postgresql.Driver.connect(Driver.java:260)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at com.j256.ormlite.jdbc.JdbcConnectionSource.makeConnection(JdbcConnectionSource.java:266)
    at com.j256.ormlite.jdbc.JdbcPooledConnectionSource.getReadWriteConnection(JdbcPooledConnectionSource.java:140)
    at com.j256.ormlite.dao.BaseDaoImpl.update(BaseDaoImpl.java:408)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:361)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:287)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:177)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

26个等待释放连接的线程在堆栈上等待该锁，例如：

"pool-4-thread-96": waiting to acquire [0x00000000c0179e18], holding [0x00000000c0b250a8]
    at com.j256.ormlite.jdbc.JdbcPooledConnectionSource.releaseConnection(JdbcPooledConnectionSource.java:168)
    at com.j256.ormlite.dao.BaseDaoImpl.create(BaseDaoImpl.java:331)
    at vgs.vigi.servlet.OrmLite.create(OrmLite.java:181)
    at vgs.vigi.servlet.CachedDao.create(CachedDao.java:126)
    at vgs.vigi.logic.Notification.sendNotification(Notification.java:491)
    at vgs.vigi.logic.Notification$1.run(Notification.java:640)
    at vgs.lib.MyTimer$2.run(MyTimer.java:103)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

更多线程正在等待释放连接

"RaspService-828": waiting to acquire [0x00000000c0179e18], holding [0x00000000c1187f88]
    at com.j256.ormlite.jdbc.JdbcPooledConnectionSource.releaseConnection(JdbcPooledConnectionSource.java:168)
    at com.j256.ormlite.dao.BaseDaoImpl.update(BaseDaoImpl.java:412)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:361)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:287)
    at vgs.vigi.ble.CmdRaspExcutor$8.exec(CmdRaspExcutor.java:318)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:182)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

尽管许多人也试图获得联系：

"RaspService-991": waiting to acquire [0x00000000c0179e18], holding [0x00000000c1624058]
    at com.j256.ormlite.jdbc.JdbcPooledConnectionSource.getReadWriteConnection(JdbcPooledConnectionSource.java:125)
    at com.j256.ormlite.dao.BaseDaoImpl.update(BaseDaoImpl.java:408)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:361)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:287)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:177)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

正在发生某些GC（inconsistent意味着线程已“阻塞（在对象监视器上）”，而没有等待任何东西）

"qtp1719311117-931": inconsistent?, holding [0x00000000eabfc510]
    at java.lang.Runtime.gc(Native Method)
    at java.lang.System.gc(System.java:993)
    at vgs.vigi.servlet.OrmLite.clearCache(OrmLite.java:33)
    at vgs.vigi.servlet.OrmLite.dao(OrmLite.java:215)
    at vgs.vigi.servlet.OrmLite.getAll(OrmLite.java:300)
    at vgs.vigi.servlet.CachedDao.getAll(CachedDao.java:227)
    at vgs.lib.Ajax.sGetAll(Ajax.java:101)
    ...

还有另一个线程中的GC（在我们的代码中显式编码-不确定为什么）

"RaspService-1882": running, holding [0x00000000c05c84c8, 0x00000000c2bed7b0]
    at java.lang.Runtime.gc(Native Method)
    at java.lang.System.gc(System.java:993)
    at vgs.vigi.servlet.OrmLite.clearCache(OrmLite.java:33)
    at vgs.vigi.servlet.OrmLite.dao(OrmLite.java:215)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:360)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:285)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:177)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

我可以期望OrmLite使用如上所述的多线程方法是安全的吗？

是否有最佳实践来避免此问题（同时仍保持服务器的多线程性质）？

---

更新

我有第二次运行的线程转储，看起来有点不同。

这里持有所有人正在等待的锁的线程不一致

"RaspService-1405": inconsistent?, holding [0x00000000c01da9b8, 0x00000000c1cfed28]

带有原始堆栈：

"RaspService-1405" #1469 prio=5 os_prio=0 tid=0x0000000021579800 nid=0xa2f4 waiting for monitor entry [0x000000002b36e000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at com.j256.ormlite.jdbc.JdbcPooledConnectionSource.getReadWriteConnection(JdbcPooledConnectionSource.java:125)
    - locked <0x00000000c01da9b8> (a java.lang.Object)
    at com.j256.ormlite.dao.BaseDaoImpl.update(BaseDaoImpl.java:408)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:361)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:287)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:177)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
    - <0x00000000c1cfed28> (a java.util.concurrent.ThreadPoolExecutor$Worker)

还有一个从连接读取的RUNNING线程。不确定是否被阻止：

"RaspService-1410": running, holding [0x00000000ed1877c8, 0x00000000c1cfe208]
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
    at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
    at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
    at org.postgresql.core.PGStream.receiveChar(PGStream.java:335)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2008)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:310)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:447)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:368)
    at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:158)
    at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:124)
    at com.j256.ormlite.jdbc.JdbcDatabaseConnection.update(JdbcDatabaseConnection.java:294)
    at com.j256.ormlite.jdbc.JdbcDatabaseConnection.update(JdbcDatabaseConnection.java:217)
    at com.j256.ormlite.stmt.mapped.MappedUpdate.update(MappedUpdate.java:101)
    at com.j256.ormlite.stmt.StatementExecutor.update(StatementExecutor.java:472)
    at com.j256.ormlite.dao.BaseDaoImpl.update(BaseDaoImpl.java:410)
    at vgs.vigi.servlet.OrmLite.update(OrmLite.java:361)
    at vgs.vigi.servlet.CachedDao.update(CachedDao.java:287)
    at vgs.vigi.ble.RaspClient.run(RaspClient.java:177)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

更新2：

这里是从pgAdmin看的会话图

Answer 1

似乎正在等待套接字：

它等待直到创建新的连接，并且还在池中保持lock。

您在Postgres中有会话限制吗？如果您这样做，建议您将其设置为比Java中的池大小稍大一些。

否则，如果池大小等于会话限制大小，则很容易出现死锁

1. 获取所有连接（达到Java池限制，达到会话数限制）
2. 应用程序尝试获取新的连接，获取池锁并被PG阻止
3. 应用程序尝试释放连接，它无法获取池锁，因此无法释放与PG的连接，因此仍达到会话限制