[File] [PATCH] Magdir/msdos DOS executable misidentfies ESP-IDF application image
Jörg Jenderek (GMX)
joerg.jen.der.ek at gmx.net
Tue Jan 9 23:12:18 UTC 2024
Hello,
some days ago i update the firmware of some IT devices (like WLED
OpenDTU). When i run file command version 5.45 with -k option on such
samples and related files i get a output like:
CPQ0TD.DRV: DOS executable (Lotus driver)
COMPAQ Text Display,
start instruction 0xe9020000 01009700
DRMOUSE.COM: DOS executable (COM),
start instruction 0xe9810500 00000000
GAG.COM: DOS executable (COM),
start instruction 0xe9fd0100 00000000
NDN.COM: DOS executable (COM),
start instruction 0xe9fe0400 00000000
USBDRIVE.COM: DOS executable (COM),
start instruction 0xe907be0d 55534244
WLED_0.14.0_ESP32-C3.bin: ESP-IDF application image for ESP32-C3,
project name: "arduino-lib-builder",
version esp-idf: v4.4.3 6407ecb3f8,
compiled on Dec 20 2022 23:49:02,
IDF version: v4.4.3,
entry address: 0x403833CE
DOS executable (COM),
start instruction 0xe906032f ce333840
WLED_0.14.0_ESP32-S2.bin: ESP-IDF application image for ESP32-S2,
project name: "arduino-lib-builder",
version esp-idf: v4.4.3 6407ecb3f8,
compiled on Dec 20 2022 23:46:13,
IDF version: v4.4.3,
entry address: 0x40027A7C
DOS executable (COM or COMBOOT 16-bit),
maybe with interrupt 22h,
start instruction 0xe906032f 7c7a0240
opendtu-generic_esp32.bin: ESP-IDF application image for ESP32,
project name: "arduino-lib-builder",
version esp-idf: v4.4.4 e8bdaf9198,
compiled on Apr 20 2023 00:53:55,
IDF version: v4.4.4,
entry address: 0x40082E5C
DOS executable (COM or COMBOOT 16-bit),
maybe with interrupt 22h,
start instruction 0xe9050220 5c2e0840
With option --extension for firmware binaries 3 byte suffix bin is not
shown. For these samples wrong mime type application/x-dosexec or
application/octet-stream is shown.
For comparison reason i also run the file format identification utility
DROID (See https://sourceforge.net/projects/droid/). Here the samples
with suffix bin are therefore described as "Binary File" by PUID fmt/208.
For comparison reason i run the file format identification utility
TrID ( See https://mark0.net/soft-trid-e.html). This identifies some
samples wrong as "Sybase iAnywhere database files" by
sybase-ianywhere-dbf.trid.xml or describe samples as "Unknown!" (See
appended
trid-v-esp.txt.gz).
The binary samples are described inside Magdir/firmware by lines like:
0 ubyte 0xE9
>32 ulelong 0xABCD5432 ESP-IDF application image
The first line checks for ESP_IMAGE_HEADER_MAGIC byte of
esp_image_header_t structure which has value 0xE9. The second test
checks for ESP_APP_DESC_MAGIC_WORD with value 0xABCD5432. That is the
magic for the esp_app_desc structure. So afterwards instead of generic
application/octet-stream mime type i show user defined one and used file
name suffix by additional lines like:
!:mime application/x-espressif-bin
!:ext bin
The samples are also described inside Magdir/msdos. The first line
checks for x86 JMP 16bit instruction by line like:
0 byte 0xe9
Unfortunately this is the same as ESP_IMAGE_HEADER_MAGIC byte. The next
line checks for 16 bit address. If address is positive then we are in
branch for forward jumps. For firmware images at offset 2 flash read
mode byte spi_mode is stored. Lowest value is 0 (SPI mode QI) and
highest value is 5 (SPI mode SLOW_READ). So firmware samples are always
matched by forward branch. That is done by second line like:
>1 leshort >-1
In third test is checked that offset must be accessible. That is also
true for firmware samples and done by line like:
>>(1.s+3) byte x
In forth test it looks for values at offset 16. For disc images this
position is number of fats. To distinguish matched disc images (which
look like DOS COM executables) the number of FAT is checked. One sub
branch checks for unrealistic low number (0) of FATs. If this succeeds
it assume it is DOS executable (like GAG.COM DRMOUSE.COM NDN.COM
CPQ0TD.DRV) and call displaying sub routine. This is done by lines like:
>>>16 ubyte =0
>>>>0 use msdos-com
For inspected firmware samples bytes here are declared as reserved and
seem to be nil. So sub routine is called wrong. So i insert additional
test before calling subroutine by check for ESP_APP_DESC_MAGIC_WORD. So
this now becomes like:
>>>16 ubyte =0
>>>>32 ulelong !0xABCD5432
>>>>>0 use msdos-com
After applying the above mentioned modifications by patches
file-5.45-msdos-esp.diff and file-5.45-firmware-esp.diff and
then the wrong duplicate message for firmware vanish.
I hope my diff file can be applied in future version of file
utility.
Unfortunately some of my binary samples are not recognized. So
Magdir/firmware probably need some more modifications.
With best wishes,
Jörg Jenderek
--
Jörg Jenderek
-------------- next part --------------
--
File mailing list
File at astron.com
https://mailman.astron.com/mailman/listinfo/file
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trid-v-esp.txt.gz
Type: application/x-gzip
Size: 415 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20240110/4ba7e6b5/attachment.bin>
-------------- next part --------------
--- file-5.45/magic/Magdir/msdos.old 2023-05-21 18:04:05.000000000 +0200
+++ file-5.45/magic/Magdir/msdos 2024-01-09 23:27:36.206413300 +0100
@@ -993,7 +993,11 @@
>>>>>>>0 use msdos-com
# JMP 16bit
0 byte 0xe9
+# display DOS executable (COM or COMBOOT 16-bit strength=40=40-0) after ESP-IDF application image (strength=40=40+0) handled by ./firmware
+#!:strength -0
# 16-bit offset; for DEBUGGING!; can be negative like: USBDRIVE.COM
+# 2h (CPQ0TD.DRV) 4FEh (NDN.COM) 581h (DRMOUSE.COM) 1FDh (GAG.COM) BE07h (USBDRIVE.COM)
+#>1 uleshort x \b, OFFSET=%#4.4x
#>1 leshort x \b, OFFSET %d
# forward jumps
>1 leshort >-1
@@ -1006,8 +1010,11 @@
>>>>0 use msdos-com
# check for unrealistic low number of FATs. Then it is not a disk image and it is a DOS executable
# like: GAG.COM DRMOUSE.COM NDN.COM CPQ0TD.DRV
+# or ESP-IDF application image like: WLED_0.14.0_ESP32-C3.bin opendtu-generic_esp32.bin
>>>16 ubyte =0
->>>>0 use msdos-com
+# skip ESP-IDF application image handled by ./firmware with ESP_APP_DESC_MAGIC_WORD
+>>>>32 ulelong !0xABCD5432
+>>>>>0 use msdos-com
# maybe disc image with valid number of FATs or DOS executable
# like: IPXODI.COM PERUSE.COM TASKID.COM
>>>16 default x
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file-5.45-msdos-esp.diff.sig
Type: application/octet-stream
Size: 860 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20240110/4ba7e6b5/attachment.obj>
-------------- next part --------------
--- file-5.45/magic/Magdir/firmware.old 2023-04-02 17:19:13.000000000 +0200
+++ file-5.45/magic/Magdir/firmware 2024-01-10 00:00:22.948554100 +0100
@@ -111,11 +111,20 @@
# ESP-IDF application image
# From: Alexandre Iooss <erdnaxe at crans.org>
+# Update: Joerg Jenderek
# URL: https://github.com/espressif/esp-idf/blob/v5.0/components/bootloader_support/include/esp_app_format.h
+# Reference: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/app_image_format.html
# Note: Concatenation of esp_image_header_t, esp_image_segment_header_t and esp_app_desc_t
# First segment contains esp_app_desc_t
+# ESP_IMAGE_HEADER_MAGIC at the beginning of esp_image_header_t structure
0 ubyte 0xE9
+# display ESP-IDF application image (strength=40=40+0) before DOS executable with 16bit JuMP (strength=40) handled by ./msdos
+#!:strength +0
+# ESP_APP_DESC_MAGIC_WORD; magic for the esp_app_desc_t structure
>32 ulelong 0xABCD5432 ESP-IDF application image
+#!:mime application/octet-stream
+!:mime application/x-espressif-bin
+!:ext bin
>>12 uleshort 0x0000 for ESP32
>>12 uleshort 0x0002 for ESP32-S2
>>12 uleshort 0x0005 for ESP32-C3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file-5.45-firmware-esp.diff.sig
Type: application/octet-stream
Size: 810 bytes
Desc: not available
URL: <https://mailman.astron.com/pipermail/file/attachments/20240110/4ba7e6b5/attachment-0001.obj>
More information about the File
mailing list