SOA Server start up issue due to low on PermGen space

ISSUE:

SOA server health was impacted because of below error.

ERROR OBSERVED:

Logs : $MW_HOME/user_projects/domain/$DOMAIN_NAME/servers/soa_server/logs/soa_server.out

<<

java.lang.OutOfMemoryError: PermGen space

java.lang.OutOfMemoryError: PermGen space

javax.ejb.EJBException: EJB encountered System Exception: : java.lang.OutOfMemoryError: PermGen space

>>

SOLUTION:

1) Permanent generation of heap is used to store String pool and various Meta data required by JVM related to Class, method and other java primitives. 

2) You can easily run out of memory if you have too many classes or huge number of Strings in your project.

3) It doesn't depends on –Xmx value so no matter how big your total heap size you can ran OutOfMemory in perm space.

4) However we specify size of permanent generation using JVM options "-XX:PermSize" and "-XX:MaxPermSize" based on project need.

Step 1) Please check if it's feasible for you to increase -XX:PermSize/-XX:MaxPermSize parameters.

(We need to check whether enough free memory is available in server through top cmd)

Eg: $ top

load averages: 1.96, 1.70, 1.45; up 121+18:33:20 08:59:31
235 processes: 180 sleeping, 1 zombie, 50 stopped, 4 on cpu
CPU states: 97.9% idle, 1.1% user, 1.0% kernel, 0.0% iowait, 0.0% swap
Memory: 64G phys mem, 19G free mem, 48G total swap, 48G free swap

Step 2) In above example it shows that 19GB free memory is available and hence it would be possible to increase the value of -XX:PermSize=768 and -XX:MaxPermSize=1280 parameter as below:

Step 3) Login to Weblogic Administration Console.

Step 4) Click Lock & Edit

Step 5) Click Servers-->soa_server-->Configuration-->Server Start-->Arguments

Current: -Xms4096m -Xmx4096m -Xmn1024m -Xss1024k -XX:PermSize=512m -XX:MaxPermSize=1024m

New: -Xms4096m -Xmx4096m -Xmn1024m -Xss1024k -XX:PermSize=768m -XX:MaxPermSize=1280m

Note: It is always advisable to increase all memory parameter in multiples of 256m.

Note : In case of cluster setup repeat above step(1,2 and 5) for another instance of SOA server to maintain consistency.

Step 6) Click Save

Step 7) Click Activate changes

Step 8) Shut down SOA Managed Server from console

Step 9) Take backup of tmp and cache directory @ $MW_HOME/user_projects/domain/$DOMAIN_NAME/servers/soa_server

mv tmp tmp_BKP

mv cache_cache_BKP

Step 10) Start SOA Managed Server (Upon startup above folders will be automatically created)

Step 11) Retest the issue.

Ref:

JVM Memory Monitoring, Tuning, Garbage Collection, Out of Memory, and Heap Dump Analysis For SOA Suite Integration 11g (Doc ID 1358719.1)