?

Log in

No account? Create an account

java pitfalls

« previous entry | next entry »
Feb. 9th, 2013 | 02:43 pm

So, I've got a java program that needs a fair amount of memory (about 2.4 GiB). Just running it the regular way causes it to exit with an OOM error; the program is smart enough to catch this and suggest adjusting the heap size by running it as java -Xmx<size> -jar <program>.jar (at the same time, it's not smart enough to suggest a reasonable size there, instead defaulting to 512M, but eh).

This works, but it's less than satisfactory, so I looked into how to set java's default heap size. A quick search turns up this page; one of the answers states that "[a]ssuming that we have more than 1gb of physical memory(quite common these days), [the default heap size]'s always 1/4th of your physical memory".

Since that would work out to 4 GiB on this machine, it wouldn't be a problem. Another answer on the same page suggests that you can find the system default by running java -XX:+PrintFlagsFinal -version, which indeed prints "uintx MaxHeapSize := 4280287232 {product}", amont other things — about what you'd expect. So far, so good.

So why does the program in question not work with that default value? This is a 64-bit java (current version) on a 64-bit machine, FWIW. Any ideas?

Link | Leave a comment | Share

Comments {6}

Winterheart

(no subject)

from: winterheart01
date: Feb. 10th, 2013 12:06 pm (UTC)
Link

Because the minimum heap size it will defaulmt set is around 64mb or so.
Therefore all Java applications that need lots of ram also set -Xms1024m -Xmx2048m

Do note that you can and probably will get a PermGen OOM because of the large use of classes that will be kept in the permanent memory./ To fix that -XX:MaxPermSize=128m

and as you need 64 MB you can set the initial PermSize to 64 MB or more:

-XX:PermSize=72m

Also note that the complete amount or ram used (heap + perm) on a 32-bit JVM must not exceed around 1.5 GB since you get a native thread error. But since you're using a 64-bit JVM this should be no problem.
Please note that setting the max heap size too big will cause major performance issues when the garbage collector runs (and needs to iterate a large memory pool) as well as setting the minmum heap size too low will cause frequent heap size increments.
It is therefore essential you try and find the proper setting in between.

Also consider using Java 7's new GC which is better at managing all this.

Reply | Thread

Schneelocke

(no subject)

from: schnee
date: Feb. 10th, 2013 12:22 pm (UTC)
Link

OK, thanks for the input. :) Actually, the thread I linked above claims that the 64 MiB default heap size has been done away with since Java 1.6.0_18, which does indeed have this in its release notes:

The default maximum heap size is half of the physical memory up to a physical memory size of 192 megabytes and otherwise one fourth of the physical memory up to a physical memory size of 1 gigabyte. For example, if your machine has 128 megabytes of physical memory, then the maximum heap size is 64 megabytes, and greater than or equal to 1 gigabyte of physical memory results in a maximum heap size of 256 megabytes. [...]

Server JVM heap configuration ergonomics are now the same as the Client, except that the default maximum heap size for 32-bit JVMs is 1 gigabyte, corresponding to a physical memory size of 4 gigabytes, and for 64-bit JVMs is 32 gigabytes, corresponding to a physical memory size of 128 gigabytes.

I'm not sure what my JVM thinks it is. If it considers itself a client JVM and defaults to a heap size limit of 1 GiB, I'd understand the behaviour I'm seeing, though — although I'd not understand the default heap size value the JVM gives when printing its various flags.

Garbage collection shouldn't be a problem, I think (the application in question is neither interactive nor time-critical in any way), but I'm still grateful for any pointers. :) Right now I'm using a maximum heap size of 2 GiB, which appears to be working nicely.

I'm also using Java 7. Is the new garbage collector the default, or do you explicitely have to ask for it to be used?

Thanks again for your help. :)

EDIT: LJ doesn't allow style attributes in comments? *headdesk*



Edited at 2013-02-10 12:24 pm (UTC)

Reply | Parent | Thread

Winterheart

(no subject)

from: winterheart01
date: Feb. 10th, 2013 05:37 pm (UTC)
Link

I believe (but my memory fails here) the JVM 7 uses the new GC by default. With older versions (for example 6) this new GC was already present but had to be enabled manually through special options.

Also you can force thte JVM into client or server mode in which it will change its GC methods and threading model.
(-server and -client, default is -client)

More info: http://www.javabeat.net/2010/08/g1-garbage-collector-in-java-7-0/

This article is a bit outdated but the info should still prove helpfull.

Reply | Parent | Thread

Schneelocke

(no subject)

from: schnee
date: Feb. 10th, 2013 05:51 pm (UTC)
Link

Cool, thanks!

BTW, is there a system-wide configuration file for Java that one could use to override the various defaults?

Reply | Parent | Thread

Winterheart

(no subject)

from: winterheart01
date: Feb. 10th, 2013 10:13 pm (UTC)
Link

you can either add these settings I believe to jvm.conf file located in the jaav (JRE or JDK) installation folder but most definitly you cazn add an JAVA_OPTS environment variable and add these options there.

However, using a large heap for all java processes is unwise, therefore it is better to keep the memory related options on a per-application basis.

Reply | Parent | Thread

Schneelocke

(no subject)

from: schnee
date: Feb. 10th, 2013 10:19 pm (UTC)
Link

Yes, good point. Again, thanks for you help! :)

Reply | Parent | Thread