c# - bash error code 137 vs 1 when out of memory -

context

i run following command in linux bash :

mono --debug --debugger-agent=transport=dt_socket,address=198.178.155.198:10000 ./stress.exe 

stress.exe c# application.

what happens

at 1 point system out of memory, wanted. error code returned.

error code returned (echo $?)

code 1 : when program creates throw because it's out of memory.

code 137 : when killed os when overloading memory.

question

why sometime os kills application? why result not same?

assuming:

• mono running sgen based gc
• linux oom killer enabled
• your stress.exe alloc'ing managed memory i.e. no native interop, no use of marshaling memory allocators, no code flagged unsafe, etc..
• you creating objects , never release refs.

lets talk sgen, alloc objects, created in nursery, run out of memory in nursery, when gc sweep , has nursery collection full, live objects move it's major heap. if major head full, more os memory requested. can adjust amount of initial memory allocated mono app , fix amount of memory (max) sgen can use. managed objects on 8000 bytes handle sgen's large object space manager , non-nursery/major-heap based memory still managed objects/memory.

so when mono needs more space managed objects , os request additional block , os says no, see outofmemory exception , 0 exit code. stress test happy.

but oom watching mono process , adjusting it's score (oom_score) higher , higher. strike mono process @ moment, put odds right @ time of gc sweep when app threads suspended sgen before sgen os memory request due out of managed memory space in nusery. exit of 137. 137 & 127 = 9, mono process sent sigkill signal (kill -9) , stress test not happy.

try experiment:

• 1) if turn off oom killer completely. assuming not live production box stressing ;-) should see "system.outofmemoryexception" 100% of time.
• or 2) set oom_adj of mono process -17 , oom leave alone. wrap mono launch in shell script grab it's pid , echo -17 process's oom_adj.
• or 3) if adjust oom_adj of mono process lower (down way -16, see mono capturing it's own managed memory outage 'more of time' never 100% of time....

this not mono and/or sgen/gc related 'issue' @ all. process consuming more , more memory subject oom killed. big fat oracle database or app/daemon has memory leak, etc.. subject kill'd.