[File] [PATCH] Issue with libmagic and floating point exceptions

[Vincent Mihalkovic]

Hi,

The "Floating point exception (core dumped)" occurred when using libmagic
internals
on software with floating point exceptions enabled (FE_INVALID).

Sample of what happens:
"""
#include <stdio.h>
#include <math.h>
#include <fenv.h>

int feenableexcept(int);
int fedisableexcept(int);

"""
int
main(int argc, char *argv[])
{
  union {
    int   i;
    float f;
  } u[2];
  fedisableexcept(FE_ALL_EXCEPT);
  feenableexcept(FE_INVALID);
  u[0].i = -1;  // NaN
  u[1].f = 1;
  printf("%d\n", u[0].f < u[1].f);
  //printf("%d\n", isless(u[0].f, u[1].f));
  return 0;
}
"""

$ gcc test.c -lm
$ ./a.out
Floating point exception (core dumped)

The proposed fix (by Paolo Andrade) added a simple check using macros
(__builtin_is{less,greater,unordered}) from math.h.

More detailed description in:
https://bugzilla.redhat.com/show_bug.cgi?id=2061557.

Vincent Mihalkovic
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.astron.com/pipermail/file/attachments/20221214/c707a2c6/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: floating-point-exception.patch
Type: text/x-patch
Size: 1313 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20221214/c707a2c6/attachment.bin>