[File] [PATCH] Magdir/images for TIM misidentifies dBase memo DBT+ wrong width
Christos Zoulas
christos at zoulas.com
Mon Oct 18 13:57:06 UTC 2021
Committed, thanks!
christos
> On Oct 16, 2021, at 1:32 PM, Jörg Jenderek <joerg.jen.der.ek at gmx.net> wrote:
>
> Hello,
>
> some times ago i handled dBase files. Some unexpected results lead me
> to inspection of Sony PlayStation bitmap images with file name
> extension TIM.
>
> When running running file command version 5.40 and later with -k
> option on such images and related files i get an output like:
>
> 16383x1-rle.tim: TIM image, 24-Bit,
> Pixel at (3,5) Size=24574x1
> 2x5.tim: TIM image,
> Pixel at (23,29) Size=1x5
> CHIMES.BIN: data
> FONT0P.TIM: TIM image, 4-Bit,
> Pixel at (0,0) Size=64x256,
> 16 CLUT Entries at (256,480)
> input.tim: TIM image, 4-Bit,
> Pixel at (640,96) Size=16x32,
> 16 CLUT Entries at (960,21)
> KAGE.TIM: TIM image, 8-Bit,
> Pixel at (0,0) Size=13x29,
> 256 CLUT Entries at (0,480)
> MAP01.TIM: TIM image, 8-Bit,
> Pixel at (768,256) Size=64x128,
> 256 CLUT Entries at (0,496)
> pal4-7x256.tim: TIM image,
> Pixel at (7,256) Size=31x64
> RGB24.TIM: TIM image, 24-Bit,
> Pixel at (0,0) Size=190x64
> SIDE06.TIM: TIM image, 15-Bit,
> Pixel at (0,0) Size=320x240
> TADA.BIN: data
> test.dbf: FoxBase+/dBase III, with memo .DBT DBF,
> 49 records * 232, update-date 97-5-15,
> at offset 257 1st record " 1One
> test.dbt: TIM image, (4702,14221)
> dBase III DBT, next free block index 16,
> 1st item "WHAT IS XBASE"
> TEX3P.TIM: TIM image, 4-Bit,
> Pixel at (0,0) Size=64x256,
> 16 CLUT Entries at (0,480)
> TOFULOGO.TIM: TIM image, 8-Bit,
> Pixel at (0,0) Size=128x256,
> 256 CLUT Entries at (0,480)
> TYPE00.TIM: TIM image, 15-Bit,
> Pixel at (0,0) Size=320x240
> VIRGIN.TIM: TIM image, 8-Bit,
> Pixel at (480,0) Size=128x256,
> 256 CLUT Entries at (0,0)
>
>
> Furthermore for TIM images only generic application/octet-stream is
> shown by option -i and with --extension option only ??? is displayed.
>
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html).
> This does not misidentifies dBase memo test.dbt as TIM image. It also
> identifies example pal4-7x256.tim correctly as 4-bit variant without
> CLUT ( colour lookup table) by definition bitmap-tim-4-nc.trid.xml and
> example 2x5.tim correctly as 8-bit variant without CLUT by
> bitmap-tim-8-nc.trid.xml. But TrID also misidentifies all TIM images as
> "LTAC compressed audio (v1.61)" with low priority by definition
> audio-ltac-161.trid.xml like the real audio examples CHIMES.BIN and
> TADA.BIN (See appended trid-v-tim.txt.gz).
> It also displays correct file name extension TIM and related URL.
> There i find also some links to samples, but the mentioned 4 and 8
> bit examples always have a Colour Look Up Table (CLUT).
>
> So first i add this information inside Magdir/images by additional
> comment lines likes:
>
> # URL: fileformats.archiveteam.org/wiki/TIM_(PlayStation_graphics)
> # Reference: https://mrclick.zophar.net/TilEd/download/timgfx.txt
>
> The recognition happened by lines like:
> 0 lelong 0x00000010 TIM image,
> >4 lelong 0x8 4-Bit,
> >4 lelong 0x9 8-Bit,
> >4 lelong 0x2 15-Bit,
> >4 lelong 0x3 24-Bit,
>
> The first 4 bytes are used for detection, but unfortunately 3 bytes
> are nil. So this pattern is not very unique enough. So more
> additional test lines are needed.
>
> At offset four 32 flag bits are stored in little endian order.
> According to documentation the first 3 bits determine the used bits
> per pixel (BPP) with the following value meanings:
> 0 4BPP
> 1 8BPP
> 2 16BPP
> 3 24BPP
> 4 Mixed Format
> That can be shown by debug line like:
> >4 lelong&0x00000007 x \b, BPP~%u
> Unfortunately the mentioned sample files links only point to 4, 8 and
> 15 bit samples. So i create other samples with help of tools like
> BMP2TIM. But i was not able to create a Mixed Format example nor an
> example with "high" (above 32768) image dimensions. I did do that
> because in the Kaitai Structure Specification for TIM images the
> integers are unsigned whereas in magic lines signed integers are
> used. So i expect trouble ( different results for "high" values" ),
> but i do if this is true. So i just keep signed integers and mention
> this fact by a comment line.
>
> The forth bit (value 8) determines if image contains a Colour Lookup
> Table (CLUT) or not. The remaining 28 bits in flag field are unused,
> that means these bits are 0 or 3 upper bytes of flag field are nil.
> That facts are used by TrID definitions bitmap-tim-*.trid.xml. For
> dBase memo example test.dbt that value is 22d10189h. That can be
> shown by debug line like:
> >4 lelong x FLAGS=%x
>
> So TrID does not misidentifies DBT example as TIM image. So i skip
> this dBase III memo example by test for unused bits of flags field by
> additional second line. So this now starts like:
>
> 0 lelong 0x00000010
> >4 lelong&0xffFFffF0 =0 Sony PlayStation bitmap image
> !:mime image/x-sony-tim
> !:ext tim
> >>4 lelong&0x07 0x0 4-Bit,
> >>4 lelong&0x07 0x1 8-Bit,
> >>4 lelong 0x2 15-Bit,
> >>4 lelong 0x3 24-Bit,
> >>4 lelong&0x07 0x4 Mixed-Bit,
>
> In file command version 5.40 the describing phrase "TIM image" was
> used. I think that is not a help for users. The users may think that
> these are images of a person like Tim Berners-Lee. So i looked how
> others name these images. ImageMagick software use phrase "PSX TIM".
> XnView graphic software use the twisted phrase "TIM PSX" and TrID use
> a describing text like "PSX TIM bitmap". If you are not a computer
> player, you also do not know what is a PSP. That is the codename for
> Sony's video game console PlayStation. I myself did not know that.
> So i now use the phrase "Sony PlayStation PSX image" instead. The TIM
> abbreviation is now shown by file name extension and by a user
> defined mime type.
> To get the correct BPP information the flags field must be unmasked
> of CLUT bit. The 16BPP and 24BPP examples have no CLUT. So the
> original expression can be kept.
>
> At offset 8 a integer value is stored as 4 byte little endian. That
> value is a sum. One part is the size of the following CLUT block for
> examples with CLUT or the image data size for examples without CLUT.
> The second part of the sum is the number (12=Ch) of prepending bytes
> with containing information like dimensions and origin points. So the
> reminder of that value seems to be always ending with hexadecimal
> value C. For misidentified DBT example this remaining value is 3.
> That information can be shown by debugging lines like:
> >8 lelong x \b, 12+CLUT or data size=%8.8x
> >8 ubyte&0x0F =0x0C \b, SIZE REMAINDER IS 12
> So this can probably be used as third possible test.
>
> For examples without CLUT directly after first size field comes image
> data block starting with X+Y origin followed by dimensions. That was
> expressed by lines like:
> >4 lelong ^8
> >>12 leshort x Pixel at (%d,
> >>14 leshort x \b%d)
> >>16 leshort x Size=%dx
> >>18 leshort x \b%d
>
> On reference site software is mentioned that can handle such images like:
> XnView
> ImageMagick
> RECOIL
> So run i command line tools to get information about images by
> commands like:
> nconvert -fullinfo *.tim > xnview.txt
> identify. *.tim > imagemagick.txt
> recoil2png -o TMP.PNG input.tim; file TMP.PNG >>recoil.txt
>
> If i look in text output i see that image height is always the same
> but the width of images reported by file command is often wrong.
>
> When looking in documentation there is written that in field before
> height not the real width width is stored. Instead the value must be
> multiplied with a scale factor to get the real image width. This
> factor depends on BPP value. So multiply by 4 (for 4BPP) 2 (for 8BPP)
> 1 (for 16BPP) and 2/3 (for 24BPP). So the branch without CLUT now
> becomes like:
> >4 lelong ^8
> >>12 leshort x Pixel at (%d,
> >>14 leshort x \b%d) Size=
> >>>4 lelong 0x0
> >>>>16 leshort*4 x \b%d
> >>>4 lelong 0x1
> >>>>16 leshort*2 x \b%d
> >>>4 lelong 0x2
> >>>>16 leshort x \b%d
> >>>4 lelong 0x3
> #>>>>16 leshort*2/3 x \b%d
> >>>>16 leshort x \b2/3*%d
> >>>4 lelong 0x4
> >>>>16 leshort x \b%d
> >>18 leshort x \bx%d
>
> Unfortunately multiplying with a fraction like 2/3 does not work. So
> i display that information in text form like for example
> 16383x1-rle.tim by phrase like "Size=2/3*24574x1".
> Then i do also scaling of width for branch with CLUT in same way.
>
> After applying the above mentioned modifications by patch
> file-images-tim.diff then now all inspected Sony PlayStation TIM
> bitmap images are described with correct BPP and dimension
> information and misidentification of dBase memo test.dbt vanish. This
> now looks like:
> 16383x1-rle.tim: Sony PlayStation PSX image, 24-Bit,
> Pixel at (3,5) Size=2/3*24574x1
> 2x5.tim: Sony PlayStation PSX image, 8-Bit,
> Pixel at (23,29) Size=2x5
> CHIMES.BIN: data
> FONT0P.TIM: Sony PlayStation PSX image, 4-Bit,
> Pixel at (0,0) Size=256x256,
> 16 CLUT Entries at (256,480)
> input.tim: Sony PlayStation PSX image, 4-Bit,
> Pixel at (640,96) Size=64x32,
> 16 CLUT Entries at (960,21)
> KAGE.TIM: Sony PlayStation PSX image, 8-Bit,
> Pixel at (0,0) Size=26x29,
> 256 CLUT Entries at (0,480)
> MAP01.TIM: Sony PlayStation PSX image, 8-Bit,
> Pixel at (768,256) Size=128x128,
> 256 CLUT Entries at (0,496)
> pal4-7x256.tim: Sony PlayStation PSX image, 4-Bit,
> Pixel at (7,256) Size=124x64
> RGB24.TIM: Sony PlayStation PSX image, 24-Bit,
> Pixel at (0,0) Size=2/3*190x64
> SIDE06.TIM: Sony PlayStation PSX image, 15-Bit,
> Pixel at (0,0) Size=320x240
> TADA.BIN: data
> test.dbf: FoxBase+/dBase III, with memo .DBT DBF,
> 49 records * 232, update-date 97-5-15,
> at offset 257 1st record " 1One
> test.dbt: dBase III DBT, next free block index 16,
> 1st item "WHAT IS XBASE"
> TEX3P.TIM: Sony PlayStation PSX image, 4-Bit,
> Pixel at (0,0) Size=256x256,
> 16 CLUT Entries at (0,480)
> TOFULOGO.TIM: Sony PlayStation PSX image, 8-Bit,
> Pixel at (0,0) Size=256x256,
> 256 CLUT Entries at (0,480)
> TYPE00.TIM: Sony PlayStation PSX image, 15-Bit,
> Pixel at (0,0) Size=320x240
> VIRGIN.TIM: Sony PlayStation PSX image, 8-Bit,
> Pixel at (480,0) Size=256x256,
> 256 CLUT Entries at (0,0)
>
> I hope my diff file can be applied in future version of file utility.
>
>
> With best wishes
> Jörg Jenderek
> --
> Jörg Jenderek
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> <xnview.txt.gz><recoil.txt.gz><imagemagick.txt.gz><trid-v-tim.txt.gz><file-images-tim_diff.DEFANGED-364><file-images-tim_diff_sig.DEFANGED-365>--
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20211018/e40d409e/attachment.asc>
More information about the File
mailing list