[File] [SECURITY] Integer Overflow in encoding.c Leading to Heap Buffer Overflow (file-5.17, 32-bit)

[Christos Zoulas]

Was fixed in 2021 by limiting the size of the buffer:

1.27         (christos 05-Feb-21):      if (nbytes > ms->encoding_max)
1.27         (christos 05-Feb-21):              nbytes = ms->encoding_max;


christos

> On Apr 1, 2026, at 11:18 AM, Kerwin <kerwinxia66001 at gmail.com> wrote:
> 
> Hi maintainers,
> 
> I am reporting an integer overflow vulnerability in libmagic (file-5.17) where the buffer size calculation `(nbytes + 1) * sizeof(unichar)` in `file_encoding()` (encoding.c:79-80) overflows on 32-bit platforms, causing `calloc` to allocate only 8 bytes while `looks_ascii()` writes past it, resulting in a heap buffer overflow.
> 
> Please find the detailed vulnerability report and proof-of-concept files attached.
> 
> Best regards
> 
> <POC.tar>-- 
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.astron.com/pipermail/file/attachments/20260417/6ce8c2e8/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://mailman.astron.com/pipermail/file/attachments/20260417/6ce8c2e8/attachment.asc>