[File] [PATCH] Magdir/msdos DOS executable misidentfies ESP-IDF application image

Sun Jan 14 17:51:52 UTC 2024

Committed, thanks!

christos

> On Jan 9, 2024, at 6:12 PM, Jörg Jenderek (GMX) <joerg.jen.der.ek at gmx.net> wrote:
> 
> Hello,
> 
> some days ago i update the firmware of some IT devices (like WLED
> OpenDTU). When i run file command version 5.45 with -k option on such
> samples and related files i get a output like:
> 
> CPQ0TD.DRV:                DOS executable (Lotus driver)
> 			   COMPAQ Text Display,
> 			   start instruction 0xe9020000 01009700
> DRMOUSE.COM:               DOS executable (COM),
> 			   start instruction 0xe9810500 00000000
> GAG.COM:                   DOS executable (COM),
> 			   start instruction 0xe9fd0100 00000000
> NDN.COM:                   DOS executable (COM),
> 			   start instruction 0xe9fe0400 00000000
> USBDRIVE.COM:              DOS executable (COM),
> 			   start instruction 0xe907be0d 55534244
> WLED_0.14.0_ESP32-C3.bin:  ESP-IDF application image for ESP32-C3,
> 			   project name: "arduino-lib-builder",
> 			   version esp-idf: v4.4.3 6407ecb3f8,
> 			   compiled on Dec 20 2022 23:49:02,
> 			   IDF version: v4.4.3,
> 			   entry address: 0x403833CE
> 			   DOS executable (COM),
> 			   start instruction 0xe906032f ce333840
> WLED_0.14.0_ESP32-S2.bin:  ESP-IDF application image for ESP32-S2,
> 			   project name: "arduino-lib-builder",
> 			   version esp-idf: v4.4.3 6407ecb3f8,
> 			   compiled on Dec 20 2022 23:46:13,
> 			   IDF version: v4.4.3,
> 			   entry address: 0x40027A7C
> 			   DOS executable (COM or COMBOOT 16-bit),
> 			   maybe with interrupt 22h,
> 			   start instruction 0xe906032f 7c7a0240
> opendtu-generic_esp32.bin: ESP-IDF application image for ESP32,
> 			   project name: "arduino-lib-builder",
> 			   version esp-idf: v4.4.4 e8bdaf9198,
> 			   compiled on Apr 20 2023 00:53:55,
> 			   IDF version: v4.4.4,
> 			   entry address: 0x40082E5C
> 			   DOS executable (COM or COMBOOT 16-bit),
> 			   maybe with interrupt 22h,
> 			   start instruction 0xe9050220 5c2e0840
> 
> With option --extension for firmware binaries 3 byte suffix bin is not
> shown. For these samples wrong mime type application/x-dosexec or
> application/octet-stream is shown.
> 
> For comparison reason i also run the file format identification utility
> DROID (See https://sourceforge.net/projects/droid/). Here the samples
> with suffix bin are therefore described as "Binary File" by PUID fmt/208.
> 
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html). This identifies some
> samples wrong as "Sybase iAnywhere database files" by
> sybase-ianywhere-dbf.trid.xml or describe samples as "Unknown!" (See
> appended
> trid-v-esp.txt.gz).
> 
> The binary samples are described inside Magdir/firmware by lines like:
> 0	ubyte		0xE9
> >32	ulelong		0xABCD5432	ESP-IDF application image
> The first line checks for ESP_IMAGE_HEADER_MAGIC byte of
> esp_image_header_t structure which has value 0xE9. The second test
> checks for ESP_APP_DESC_MAGIC_WORD with value 0xABCD5432. That is the
> magic for the esp_app_desc structure. So afterwards instead of generic
> application/octet-stream mime type i show user defined one and used file
> name suffix by additional lines like:
> !:mime	application/x-espressif-bin
> !:ext	bin
> 
> The samples are also described inside Magdir/msdos. The first line
> checks for x86 JMP 16bit instruction by line like:
> 0           byte    0xe9
> Unfortunately this is the same as ESP_IMAGE_HEADER_MAGIC byte. The next
> line checks for 16 bit address. If address is positive then we are in
> branch for forward jumps. For firmware images at offset 2 flash read
> mode byte spi_mode is stored. Lowest value is 0 (SPI mode QI) and
> highest value is 5 (SPI mode SLOW_READ). So firmware samples are always
> matched by forward branch. That is done by second line like:
> >1		leshort	>-1
> In third test is checked that offset must be accessible. That is also
> true for firmware samples and done by line like:
> >>(1.s+3)   byte    x
> In forth test it looks for values at offset 16. For disc images this
> position is number of fats. To distinguish matched disc images (which
> look like DOS COM executables) the number of FAT is checked. One sub
> branch checks for unrealistic low number (0) of FATs. If this succeeds
> it assume it is DOS executable (like GAG.COM DRMOUSE.COM NDN.COM
> CPQ0TD.DRV) and call displaying sub routine. This is done by lines like:
> >>>16		ubyte		=0
> >>>>0			use		msdos-com
> For inspected firmware samples bytes here are declared as reserved and
> seem to be nil. So sub routine is called wrong. So i insert additional
> test before calling subroutine by check for ESP_APP_DESC_MAGIC_WORD. So
> this now becomes like:
> >>>16		ubyte		=0
> >>>>32	ulelong		!0xABCD5432
> >>>>>0			use		msdos-com
> 
> After applying the above mentioned modifications by patches
> file-5.45-msdos-esp.diff and file-5.45-firmware-esp.diff and
> then the wrong duplicate message for firmware vanish.
> 
> I hope my diff file can be applied in future version of file
> utility.
> 
> Unfortunately some of my binary samples are not recognized. So
> Magdir/firmware probably need some more modifications.
> 
> With best wishes,
> Jörg Jenderek
> --
> Jörg Jenderek
> <Nachrichtenteil als Anhang.DEFANGED-0><trid-v-esp.txt.gz><file-5_45-msdos-esp_diff.DEFANGED-1><file-5_45-msdos-esp_diff_sig.DEFANGED-2><file-5_45-firmware-esp_diff.DEFANGED-3><file-5_45-firmware-esp_diff_sig.DEFANGED-4>-- 
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>