[wtn]

Since 078020e, HEAD_BYTE_REQUIRED, the head_byte value for "nothing cached", is 0xc1, the one first byte the spec marks "(never used)". A 0xc1 read from the input is cached as the same value, so the next get_head_byte thinks the cache is empty and reads past it.

skip_nil returns false and leaves the 0xc1 cached. read_array_header and read_map_header raise UnexpectedTypeError and leave it cached too. The next read then skips the invalid byte:

u = MessagePack::Unpacker.new
u.feed("\xc1\x01".b)
u.skip_nil  #=> false
u.read      #=> 1, should raise MalformedFormatError

read and skip aren't affected; read_primitive rejects 0xc1 first.

Fix

HEAD_BYTE_REQUIRED is now 0x100. head_byte is an unsigned int and only ever receives 0..255 from the input, so the sentinel can't collide. It is only assigned in init, reset and reset_head_byte, and only compared in get_head_byte. JRuby has its own unpacker.

The JRuby decoder already behaves this way: read_array_header and read_map_header restore the buffer position on error, and skip_nil is not implemented there.

The new specs call skip_nil, read_array_header or read_map_header on "\xc1\x01" and check that the next read raises MalformedFormatError. Without the change they fail with expected MessagePack::MalformedFormatError but nothing was raised.