[File] [PATCH] Magdir/images for TIM misidentifies dBase memo DBT+ wrong width

Christos Zoulas christos at zoulas.com
Mon Oct 18 13:57:06 UTC 2021


Committed, thanks!

christos

> On Oct 16, 2021, at 1:32 PM, Jörg Jenderek <joerg.jen.der.ek at gmx.net> wrote:
> 
> Hello,
> 
> some times ago i handled dBase files. Some unexpected results lead me
> to inspection of Sony PlayStation bitmap images with file name
> extension TIM.
> 
> When running running file command version 5.40 and later with -k
> option on such images and related files i get an output like:
> 
> 16383x1-rle.tim:  TIM image, 24-Bit,
> 		  Pixel at (3,5) Size=24574x1
> 2x5.tim:          TIM image,
> 		  Pixel at (23,29) Size=1x5
> CHIMES.BIN:       data
> FONT0P.TIM:       TIM image, 4-Bit,
> 		  Pixel at (0,0) Size=64x256,
> 		  16 CLUT Entries at (256,480)
> input.tim:        TIM image, 4-Bit,
> 		  Pixel at (640,96) Size=16x32,
> 		  16 CLUT Entries at (960,21)
> KAGE.TIM:         TIM image, 8-Bit,
> 		  Pixel at (0,0) Size=13x29,
> 		  256 CLUT Entries at (0,480)
> MAP01.TIM:        TIM image, 8-Bit,
> 		  Pixel at (768,256) Size=64x128,
> 		  256 CLUT Entries at (0,496)
> pal4-7x256.tim:   TIM image,
> 		  Pixel at (7,256) Size=31x64
> RGB24.TIM:        TIM image, 24-Bit,
> 		  Pixel at (0,0) Size=190x64
> SIDE06.TIM:       TIM image, 15-Bit,
> 		  Pixel at (0,0) Size=320x240
> TADA.BIN:         data
> test.dbf:         FoxBase+/dBase III, with memo .DBT DBF,
> 		  49 records * 232, update-date 97-5-15,
> 		  at offset 257 1st record "       1One
> test.dbt:         TIM image, (4702,14221)
> 		  dBase III DBT, next free block index 16,
> 		  1st item "WHAT IS XBASE"
> TEX3P.TIM:        TIM image, 4-Bit,
> 		  Pixel at (0,0) Size=64x256,
> 		  16 CLUT Entries at (0,480)
> TOFULOGO.TIM:     TIM image, 8-Bit,
> 		  Pixel at (0,0) Size=128x256,
> 		  256 CLUT Entries at (0,480)
> TYPE00.TIM:       TIM image, 15-Bit,
> 		  Pixel at (0,0) Size=320x240
> VIRGIN.TIM:       TIM image, 8-Bit,
> 		  Pixel at (480,0) Size=128x256,
> 		  256 CLUT Entries at (0,0)
> 
> 
> Furthermore for TIM images only generic application/octet-stream is
> shown by option -i and with --extension option only ??? is displayed.
> 
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html).
> This does not misidentifies dBase memo test.dbt as TIM image. It also
> identifies example pal4-7x256.tim correctly as 4-bit variant without
> CLUT ( colour lookup table) by definition bitmap-tim-4-nc.trid.xml and
> example 2x5.tim correctly as 8-bit variant without CLUT by
> bitmap-tim-8-nc.trid.xml. But TrID also misidentifies all TIM images as
> "LTAC compressed audio (v1.61)" with low priority by definition
> audio-ltac-161.trid.xml like the real audio examples CHIMES.BIN and
> TADA.BIN (See appended trid-v-tim.txt.gz).
> It also displays correct file name extension TIM and related URL.
> There i find also some links to samples, but the mentioned 4 and 8
> bit examples always have a Colour Look Up Table (CLUT).
> 
> So first i add this information inside Magdir/images by additional
> comment lines likes:
> 
> # URL:	fileformats.archiveteam.org/wiki/TIM_(PlayStation_graphics)
> # Reference:	https://mrclick.zophar.net/TilEd/download/timgfx.txt
> 
> The recognition happened by lines like:
> 0		lelong		0x00000010	TIM image,
> >4		lelong  	0x8		4-Bit,
> >4		lelong  	0x9		8-Bit,
> >4		lelong  	0x2		15-Bit,
> >4		lelong  	0x3		24-Bit,
> 
> The first 4 bytes are used for detection, but unfortunately 3 bytes
> are nil. So this pattern is not very unique enough. So more
> additional test lines are needed.
> 
> At offset four 32 flag bits are stored in little endian order.
> According to documentation the first 3 bits determine the used bits
> per pixel (BPP) with the following value meanings:
> 	0	4BPP
> 	1	8BPP
> 	2	16BPP
> 	3	24BPP
> 	4	Mixed Format
> That can be shown by debug line like:
> >4		lelong&0x00000007	x	\b, BPP~%u
> Unfortunately the mentioned sample files links only point to 4, 8 and
> 15 bit samples. So i create other samples with help of tools like
> BMP2TIM. But i was not able to create a Mixed Format example nor an
> example with "high" (above 32768) image dimensions. I did do that
> because in the Kaitai Structure Specification for TIM images the
> integers are unsigned whereas in magic lines signed integers are
> used. So i expect trouble ( different results for "high" values" ),
> but i do if this is true. So i just keep signed integers and mention
> this fact by a comment line.
> 
> The forth bit (value 8) determines if image contains a Colour Lookup
> Table (CLUT) or not. The remaining 28 bits in flag field are unused,
> that means these bits are 0 or 3 upper bytes of flag field are nil.
> That facts are used by TrID definitions bitmap-tim-*.trid.xml. For
> dBase memo example test.dbt that value is 22d10189h. That can be
> shown by debug line like:
> >4		lelong  	x		FLAGS=%x
> 
> So TrID does not misidentifies DBT example as TIM image. So i skip
> this dBase III memo example by test for unused bits of flags field by
> additional second line. So this now starts like:
> 
> 0	lelong		0x00000010
> >4	lelong&0xffFFffF0	=0	Sony PlayStation bitmap image
> !:mime	image/x-sony-tim
> !:ext	tim
> >>4	lelong&0x07	0x0		4-Bit,
> >>4	lelong&0x07	0x1		8-Bit,
> >>4	lelong  	0x2		15-Bit,
> >>4	lelong  	0x3		24-Bit,
> >>4	lelong&0x07	0x4		Mixed-Bit,
> 
> In file command version 5.40 the describing phrase "TIM image" was
> used. I think that is not a help for users. The users may think that
> these are images of a person like Tim Berners-Lee. So i looked how
> others name these images. ImageMagick software use phrase "PSX TIM".
> XnView graphic software use the twisted phrase "TIM PSX" and TrID use
> a describing text like "PSX TIM bitmap". If you are not a computer
> player, you also do not know what is a PSP. That is the codename for
> Sony's video game console PlayStation. I myself did not know that.
> So i now use the phrase "Sony PlayStation PSX image" instead. The TIM
> abbreviation is now shown by file name extension and by a user
> defined mime type.
> To get the correct BPP information the flags field must be unmasked
> of CLUT bit. The 16BPP and 24BPP examples have no CLUT. So the
> original expression can be kept.
> 
> At offset 8 a integer value is stored as 4 byte little endian. That
> value is a sum. One part is the size of the following CLUT block for
> examples with CLUT or the image data size for examples without CLUT.
> The second part of the sum is the number (12=Ch) of prepending bytes
> with containing information like dimensions and origin points. So the
> reminder of that value seems to be always ending with hexadecimal
> value C. For misidentified DBT example this remaining value is 3.
> That information can be shown by debugging lines like:
> >8	lelong		x	\b, 12+CLUT or data size=%8.8x
> >8	ubyte&0x0F	=0x0C	\b, SIZE REMAINDER IS 12
> So this can probably be used as third possible test.
> 
> For examples without CLUT directly after first size field comes image
> data block starting with X+Y origin followed by dimensions. That was
> expressed by lines like:
> >4		lelong		^8
> >>12		leshort		x		Pixel at (%d,
> >>14		leshort		x		\b%d)
> >>16		leshort		x		Size=%dx
> >>18		leshort		x		\b%d
> 
> On reference site software is mentioned that can handle such images like:
> 	XnView
> 	ImageMagick
> 	RECOIL
> So run i command line tools to get information about images by
> commands like:
> nconvert -fullinfo	*.tim			> xnview.txt
> identify. 		*.tim			> imagemagick.txt
> recoil2png -o TMP.PNG input.tim; file TMP.PNG	>>recoil.txt
> 
> If i look in text output i see that image height is always the same
> but the width of images reported by file command is often wrong.
> 
> When looking in documentation there is written that in field before
> height not the real width width is stored. Instead the value must be
> multiplied with a scale factor to get the real image width. This
> factor depends on BPP value. So multiply by 4 (for 4BPP) 2 (for 8BPP)
> 1 (for 16BPP) and 2/3 (for 24BPP). So the branch without CLUT now
> becomes like:
> >4		lelong		^8
> >>12		leshort		x		Pixel at (%d,
> >>14		leshort		x		\b%d) Size=
> >>>4		lelong  	0x0
> >>>>16		leshort*4	x		\b%d
> >>>4		lelong  	0x1
> >>>>16		leshort*2	x		\b%d
> >>>4		lelong  	0x2
> >>>>16		leshort		x		\b%d
> >>>4		lelong  	0x3
> #>>>>16		leshort*2/3	x		\b%d
> >>>>16		leshort		x		\b2/3*%d
> >>>4		lelong  	0x4
> >>>>16		leshort		x		\b%d
> >>18		leshort		x		\bx%d
> 
> Unfortunately multiplying with a fraction like 2/3 does not work. So
> i display that information in text form like for example
> 16383x1-rle.tim by phrase like "Size=2/3*24574x1".
> Then i do also scaling of width for branch with CLUT in same way.
> 
> After applying the above mentioned modifications by patch
> file-images-tim.diff then now all inspected Sony PlayStation TIM
> bitmap images are described with correct BPP and dimension
> information and misidentification of dBase memo test.dbt vanish. This
> now looks like:
> 16383x1-rle.tim: Sony PlayStation PSX image, 24-Bit,
> 		 Pixel at (3,5) Size=2/3*24574x1
> 2x5.tim:         Sony PlayStation PSX image, 8-Bit,
> 		 Pixel at (23,29) Size=2x5
> CHIMES.BIN:      data
> FONT0P.TIM:      Sony PlayStation PSX image, 4-Bit,
> 		 Pixel at (0,0) Size=256x256,
> 		 16 CLUT Entries at (256,480)
> input.tim:       Sony PlayStation PSX image, 4-Bit,
> 		 Pixel at (640,96) Size=64x32,
> 		 16 CLUT Entries at (960,21)
> KAGE.TIM:        Sony PlayStation PSX image, 8-Bit,
> 		 Pixel at (0,0) Size=26x29,
> 		 256 CLUT Entries at (0,480)
> MAP01.TIM:       Sony PlayStation PSX image, 8-Bit,
> 		 Pixel at (768,256) Size=128x128,
> 		 256 CLUT Entries at (0,496)
> pal4-7x256.tim:  Sony PlayStation PSX image, 4-Bit,
> 		 Pixel at (7,256) Size=124x64
> RGB24.TIM:       Sony PlayStation PSX image, 24-Bit,
> 		 Pixel at (0,0) Size=2/3*190x64
> SIDE06.TIM:      Sony PlayStation PSX image, 15-Bit,
> 		 Pixel at (0,0) Size=320x240
> TADA.BIN:        data
> test.dbf:        FoxBase+/dBase III, with memo .DBT DBF,
> 		 49 records * 232, update-date 97-5-15,
> 		 at offset 257 1st record "       1One
> test.dbt:        dBase III DBT, next free block index 16,
> 		 1st item "WHAT IS XBASE"
> TEX3P.TIM:       Sony PlayStation PSX image, 4-Bit,
> 		 Pixel at (0,0) Size=256x256,
> 		 16 CLUT Entries at (0,480)
> TOFULOGO.TIM:    Sony PlayStation PSX image, 8-Bit,
> 		 Pixel at (0,0) Size=256x256,
> 		 256 CLUT Entries at (0,480)
> TYPE00.TIM:      Sony PlayStation PSX image, 15-Bit,
> 		 Pixel at (0,0) Size=320x240
> VIRGIN.TIM:      Sony PlayStation PSX image, 8-Bit,
> 		 Pixel at (480,0) Size=256x256,
> 		 256 CLUT Entries at (0,0)
> 
> I hope my diff file can be applied in future version of file utility.
> 
> 
> With best wishes
> Jörg Jenderek
> --
> Jörg Jenderek
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> <xnview.txt.gz><recoil.txt.gz><imagemagick.txt.gz><trid-v-tim.txt.gz><file-images-tim_diff.DEFANGED-364><file-images-tim_diff_sig.DEFANGED-365>--
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20211018/e40d409e/attachment.asc>


More information about the File mailing list