It is a bug!!!

So to find out if it is a BUG, I asked the official people. SUN. The origin of Java. The Write Once, Run anywhere language. So I raise a support request with Sun Support. Although it is not an critical issue, it is still an issue. It may even be a bug.

I email Sun Support with my problem description, the server logs, the monitoring data and the domain.xml early this week. After an exchange of email to get more information, I got my first explanation from Sun.

I was utterly disappointed with the quality. And this is why.
1) They send me an email with the explanation in plain text. The text was so badly formated that it was unreadable. The sentence was chopped off at inappropriate location. Words was broken such that it did not make sense. (It is not a bad case of Justification gone wrong, it is horrible formating.) The points did not flow. ( As in point 1, 2, 3, 5.... what happen to the 4) The paragraph has no indentation and there is reference to non existent information (eg P15).

I tried to adjust the page size but to no avail.

2) So I replied that I could not understand. So he "here by send the document in word format". That was his exact words. I expected better formatting. But alast........... the same thing. What he has done was to cut and past the text into the word doc. Atrocious formating, incomprehensible English and all. No effort to clean up the formating at all. There was even a few cases of symbols which is evidence that the article was cut and paste from somewhere where the character encoding did not match.

Is it a case of Open Office converting to MS Words? I doubt so. The plain text in the first place already say it all.

3) Lastly, with a colleague we attempt to decipher his text. We found out, it did not answer the question. The text was a rip off from somewhere informing that there is a problem with Sun Java Application Server's connection pool having an issue if there is a firewall between the app server and the db. The text also try to explain how the connection pooling mechanism work.


Conclusion:
After a discussion with another colleague, we decided that it is a bug. Under no condition should the application server create a pool that is more than the maximum pool size specified in the domain.xml.

The person who answered was from XXXXX, XXXXXXX.

Although I still support Java and Sun MicroSystems, this incident is really shaking my confidence. This is why:


AS 8.x JDBC Connection Pool
===========================
Let take the following configurable Connection pool properties from AS8.x
e="
oracle.jdbc.pool.OracleDataSource" fail-all-connections="false" idle-timeout-in-
seconds="300"
is-connection-validation-required="true" is-isolation-level-guaranteed="false" m
ax-pool-size="32" max-waittime-
in-millis="60000" name="oracle" pool-resize-quantity="2" res-type="javax.sql.Dat
aSource" steadypool-
size="8" validation-table-name="dual">




In this pool setting in domain.xml, you will notice some of the pool attributes
1. is-connection-validation-required, connection-validation-method, validation-t
able-name
2. fail-all-connections
3. idle-timeout-in-seconds
5. max-pool-size, max-wait-time-in-millis, pool-resize-quantity, steady-pool-siz
e
Connection pool behaviour before P15
1. If . is-connection-validation-required is true,
a. And if connection-validation-method can be table. In the case, before t
he JDBC connection
is returned to the application (when asked by calls like DataSource.getConnectio
n()), the
connection is checked that is it is valid by select count(*) from table. N
ot that the table is
configured by validation-table-name. This is the recommended validation fo
r Oracle and the
table name is normally DUAL
b. If connection-validation-method is auto-commit, then connection is test
ed by sequences of
calls to Connection.setAutoCommit(), getAutoCommit() and isAutoCommit(). It has
been
reported that for JDBC drivers like Oracle, this method of testing is not reliab
le (hence table
validation is recommended for Oracle).
c. If the connection-validation-method is metadata, the database Connectio
n
DatabaseMetaData query is used to test connection is valid. If one notice that t
his is used by
HADB JDBC driver.
2. Next, the fail-all-connections is a flag to indicate that if the while
taking a connection from the pool
and it is detected or encountered an exception, then all the connections in the
pool will be failed (if this
attribute is true).
Normally, fail-all-connections should be false since all the other connections i
n the pool might still be
valid and instead it would be more drastic to fail all of them due to that conne
ction.
3. Next, the idle-timeout-in-seconds is an attribute of how long the connection
can be in the pool if it is
idle. Note that implicitly, there is a background thread that is scheduled every
idle-timeout-inseconds
to do operation on pool cleanup. We will discuss this next.
4. The pool size is governed by steady-pool-size and max-pool-size.
a. Initially when the system is started the pool will be empty. On the first req
uest to ask for a
database connection, in P13, a new connection will be created up to the steady p
ool size.
b. When a new connection is created and all the other other connections in the s
teady pool size is
already taken up (ie there is no free connections in the pool), then ONE connect
ion will be
created.
c. So lets take an example,
Steady pool size is 0, and the max-pool-size is 32 and the pool-resize-quantity
is 2, when you
ask for a new connection, and return the connection, the pool should now contain
1
CONNECTION (for idle-timeout-in-seconds).
So after for idle-timeout-in-seconds, when the JDBC cleanup thread runs, this co
nnection will
be cleared and the pool size will be 0 (since the steady pool is denoted 0)
d. Now as for the max-pool-size, it is obviously binds the number of connections
this pool will
create.

5. The role of the JDBC cleanup thread in P13 is
a. Periodically wakes up idle-timeout-in-seconds
b. For any connections in the pool that exceed the steady pool size, and for the
se connections if
they are idle > idle-timeout-in-seconds, destroy them (so that they poo
l goes down to steady
pool size). Note that the number of idle connections that is clear is BOUNDED to
be only
pool-resize-quantity
c. Note that the cleanup thread also tries to maintain the pool to be steady-poo
l-size.
d. However, do note that connections that is inside the steady-pool remains in t
he pool. The
implication is that these connections can be prone to firewall timeout. The fact
is that if these
connections in the steady pool are not touched, the firewall may decide to timeo
ut the TCP
connections. If that happens, bad behaviour will happen when these connections a
re validated.


Connection pool behaviour at P15/16/17
=======================================
Now all the attributes in the above are still there. However there is some subtl
e behaviour.
In P15, an RFE was implemented so that connections in the steady pool those are
idle more than idletimeout-
seconds will be destroyed. So the following is what the cleanup thread does
1 The role of the JDBC cleanup thread in P15 is
a. Periodically wakes up idle-timeout-in-seconds
b. For all connections in the pool that are idle idle-timeout-in-seconds
, destroy them (so that
they pool goes down to steady pool size). Note that the number of idle connectio
ns that is clear is
BOUNDED to be only pool-resize-quantity. Note that ALL idle connections AR
E destroyed.
c. Note that the cleanup thread also tries to maintain the pool to be steady-poo
l-size. This implies too
that if the above connections is destroyed and less than steady-pool-size then a
number of
connections is created (up to the steady pool size)
Due to this behaviour, the connections in the pool should be firewall friendly a
s long as the “idle-timeout-inseconds”
is well below the firewall timeout (Typically that means for a firewall friendly
timeout the value of
idle-timeout-in-seconds should be firewall timeout.
2 Next, there is also some subtle behaviour to the way “pool-resize-quantity
” means. In P15, this parameter
applies not only to resize the pool downward to the steady pool size but applies
too in terms of growing
the number of connections in the pool.
Take the same example, Steady pool size is 0, and the max-pool-size is 32 and th
e pool-resize-quantity is
2, when you ask for a new connection, and return the connection, the pool will h
ave 2 CONNECTIONS
(since the implementation will grow them at pool-resize-quantity everytime.).
Now, due to the new cleanup thread and the pool resize behaviour, you probably n
otice that having a low
“idle-timeout-seconds” may cause an extremely many database connection creat
ion calls (since idle
connections will be destroy

No comments: