[File] [SECURITY] Integer Overflow in encoding.c Leading to Heap Buffer Overflow (file-5.17, 32-bit)

[Kerwin]

Hi maintainers,

I am reporting an integer overflow vulnerability in libmagic (file-5.17)
where the buffer size calculation `(nbytes + 1) * sizeof(unichar)` in
`file_encoding()` (encoding.c:79-80) overflows on 32-bit platforms, causing
`calloc` to allocate only 8 bytes while `looks_ascii()` writes past it,
resulting in a heap buffer overflow.

Please find the detailed vulnerability report and proof-of-concept files
attached.

Best regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.astron.com/pipermail/file/attachments/20260401/dd0a766e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: POC.tar
Type: application/x-tar
Size: 18944 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20260401/dd0a766e/attachment-0001.tar>