Incorrect behaviour between tables in luajit and lua

[Eduardo Barthel]

While using LuaJIT in my apps I found this bug while declaring a new table
with "nil" value in the middle, read the following examples, while in Lua
the output is 3 in LuajIT is 1.

Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3

***@archrox:~ $ lua5.1
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3

***@archrox:~ $ luajit
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,2}
print(#a)

1

[Szabó Antal]

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new table
with "nil" value in the middle, read the following examples, while in Lua
the output is 3 in LuajIT is 1.

This is not a bug, see: http://www.lua.org/manual/5.1/manual.html#2.5.5

"The length of a table t is defined to be any integer index n such
that t[n] is not nil and t[n+1] is nil; moreover, if t[1] is nil, n
can be zero. For a regular array, with non-nil values from 1 to a
given n, its length is exactly that n, the index of its last value. If
the array has "holes" (that is, nil values between other non-nil
values), then #t can be any of the indices that directly precedes a
nil value (that is, it may consider any such nil value as the end of
the array)."

[Tudor Bosman]

From http://www.lua.org/manual/5.1/manual.html#2.5.5

"The length of a table t is defined to be any integer index n such that
t[n] is not nil and t[n+1] is nil; moreover, if t[1] is nil, n can be zero.
For a regular array, with non-nil values from 1 to a given n, its length is
exactly that n, the index of its last value. If the array has "holes" (that
is, nil values between other non-nil values), then #t can be any of the
indices that directly precedes a nil value (that is, it may consider any
such nil value as the end of the array)."

Both behaviors are correct.

-Tudor.

[Alex]

This is allowed by the Lua spec [1]:

The length of a table t is defined to be any integer index n such that
t[n] is not nil and t[n+1] is nil; moreover, if t[1] is nil, n can be zero.
For a regular array, with non-nil values from 1 to a given n, its length is
exactly that n, the index of its last value. If the array has "holes" (that
is, nil values between other non-nil values), then #t can be any of the
indices that directly precedes a nil value (that is, it may consider any
such nil value as the end of the array).

[1]: http://www.lua.org/manual/5.1/manual.html#2.5.5

[Coda Highland]

This isn't a bug at all. You're invoking undefined behavior -- # is
only well-defined for sequences, and a sequence is defined as not
containing embedded nils.

/s/ Adam

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new table
with "nil" value in the middle, read the following examples, while in Lua
the output is 3 in LuajIT is 1.
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,2}
print(#a)

1

[Stefano]

And I was going to reply as well....

Post by Coda Highland
This isn't a bug at all. You're invoking undefined behavior -- # is
only well-defined for sequences, and a sequence is defined as not
containing embedded nils.
/s/ Adam

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new table
with "nil" value in the middle, read the following examples, while in Lua
the output is 3 in LuajIT is 1.
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,2}
print(#a)

1

[Eduardo Barthel]

I see, but now look this case, another different behaviour

***@archrox:~ $ lua
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,3}
pcall(print, unpack(a))

1 nil 3

***@archrox:~ $ luajit
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,3}
pcall(print, unpack(a))

1

And I was going to reply as well....

Post by Coda Highland
This isn't a bug at all. You're invoking undefined behavior -- # is
only well-defined for sequences, and a sequence is defined as not
containing embedded nils.
/s/ Adam

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new

table

Post by Coda Highland

Post by Eduardo Barthel
with "nil" value in the middle, read the following examples, while in

Lua

Post by Coda Highland

Post by Eduardo Barthel
the output is 3 in LuajIT is 1.
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink

fuse

Post by Coda Highland

Post by Eduardo Barthel

a = {1,nil,2}
print(#a)

1

[Szabó Antal]

Of course, as unpack uses the same definition of length as the #operator.

From http://www.lua.org/manual/5.1/manual.html#pdf-unpack

"By default, i is 1 and j is the length of the list, as defined by the
length operator"

I see, but now look this case, another different behaviour
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,3}
pcall(print, unpack(a))

1 nil 3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,3}
pcall(print, unpack(a))

1

And I was going to reply as well....

Post by Coda Highland
This isn't a bug at all. You're invoking undefined behavior -- # is
only well-defined for sequences, and a sequence is defined as not
containing embedded nils.
/s/ Adam

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new table
with "nil" value in the middle, read the following examples, while in Lua
the output is 3 in LuajIT is 1.
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,2}
print(#a)

1

[Eduardo Barthel]

I see that unpack uses the length operator again, but shouldn't luajit
behave like lua in such cases?

Post by Eduardo Barthel
I see, but now look this case, another different behaviour
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,3}
pcall(print, unpack(a))

1 nil 3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse

a = {1,nil,3}
pcall(print, unpack(a))

1
And I was going to reply as well....

Post by Coda Highland
This isn't a bug at all. You're invoking undefined behavior -- # is
only well-defined for sequences, and a sequence is defined as not
containing embedded nils.
/s/ Adam

Post by Eduardo Barthel
While using LuaJIT in my apps I found this bug while declaring a new

table

Post by Coda Highland

Post by Eduardo Barthel
with "nil" value in the middle, read the following examples, while in

Lua

Post by Coda Highland

Post by Eduardo Barthel
the output is 3 in LuajIT is 1.
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio

a = {1,nil,2}
print(#a)

3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc

sink fuse

Post by Coda Highland

Post by Eduardo Barthel

a = {1,nil,2}
print(#a)

1

[Coda Highland]

Post by Eduardo Barthel
I see that unpack uses the length operator again, but shouldn't luajit
behave like lua in such cases?

It DOES behave like Lua. You're invoking undefined behavior by using a
table with a nil in it. Undefined behavior has NO guarantees about its
behavior.

Even Lua will have different behaviors there depending on how you
construct that table. It's NOT guaranteed that #a will always return 3
for the table you've quoted.

/s/ Adam

[Coda Highland]

Post by Coda Highland

Post by Eduardo Barthel
I see that unpack uses the length operator again, but shouldn't luajit
behave like lua in such cases?

It DOES behave like Lua. You're invoking undefined behavior by using a
table with a nil in it. Undefined behavior has NO guarantees about its
behavior.
Even Lua will have different behaviors there depending on how you
construct that table. It's NOT guaranteed that #a will always return 3
for the table you've quoted.
/s/ Adam

Case in point, using stock Lua 5.2:

b = { }
b[1] = 1
b[2] = nil
b[3] = 2
print(#b) -- outputs 1, even though b contains { 1, nil, 2 }

/s/ Adam

[Craig Barnes]

Post by Eduardo Barthel
I see that unpack uses the length operator again, but shouldn't luajit
behave like lua in such cases?

Nope. You should just never use the length operator or unpack on
tables with holes.