PANIC in lj_cdata_setfin() on out of memory

[Roman Tsisyk]

Hi,

LuaJIT panics if there is no more memory available to resize internal finalizers table:

```
local ffi = require('ffi')
function on_gc(t)
end

local result = {}
local i = 1
while true do
   -- Make a lot void* objects with finalizers
    local t = ffi.gc(ffi.cast('void *', NULL), on_gc)
    result[i] = t
    i = i + 1
    if i % 100000 == 0 then
        print('i', i)
    end
end
```

```
luajit test.lua
...
i 134100000
i 134200000
PANIC: unprotected error in call to Lua API (not enough memory)
```

v2.0.3-37-g6d0654d

We found problem code and added a breakpoint:

```
#1  0x00000000004fa683 in lj_err_throw ()
#2  0x00000000004fa8c3 in lj_err_mem ()
#3  0x00000000004f9be2 in lj_mem_realloc ()
#4  0x00000000004fc5fc in resizetab ()
#5  0x00000000004fc8a8 in rehashtab ()
#6  0x00000000004fc95d in lj_tab_newkey ()
#7  0x000000000050d8fe in lj_cdata_setfin ()
#8  0x000000000bdaf95e in TRACE_6 () at (string):58
#9  0x00000000004a0569 in lbox_call (L=0x400fb820, nargs=3, nreturns=-1) at /home/alyapunov/Work/tarantool/src/lua/utils.h:442
```

Thanks to https://github.com/alyapunov for debugging this problem.

[Alex]

Note that this is documented in http://luajit.org/status.html

Currently some out-of-memory errors from on-trace code are not handled

correctly.

The error may fall through an on-trace pcall or it may be passed on to

the function

set with lua_atpanic on x64. This issue will be fixed with the new

garbage collector.