The LXO file format is derived from the metaformat for binary files
described in "EA IFF 85 Standard for Interchange Format Files."[4]
Mainly consisting of chunks and subchunks.
While parsing subchunks, the function Swap4 in valet4.dll takes a
length and an input buffer and proceeds to reverse DWORDs in the input
buffer for proper endianness.
A vulnerability was observed in the case of the CHNL subchunk in which
passing an invalid length to the Swap4 function would reverse every
DWORD in the stack, both reversing SEH pointer near the bottom of the
stack AND causing an exception (ie: forcing a call to the now reversed
SEH pointer).
We belive this condition may be exploitable in some scenarios as long
as the address of function __except_handler3 in kernel32.dll has a
least significant byte < 0x7F.
Proof of concept: Here is a 464 bytes long LXO file demonstrating the
issue