[File] [PATCH] Magdir/images, intel for Atari DEGAS bitmap *.pi1 *.pi2 *.pi3 *.pc1 *.pc2 *.pc3

Christos Zoulas christos at zoulas.com
Fri Dec 10 18:29:22 UTC 2021


Committed, thanks!

christos

> On Dec 8, 2021, at 8:07 PM, Jörg Jenderek <joerg.jen.der.ek at gmx.net> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hello,
> some times ago i send patches for lif file because the starting 2
> byte magic 8000h was too weak. Unfortunately this starting sequence
> also occur for Atari DEGAS low-resolution bitmap (*.pc1).
> Now i inspect such Atari images with 6 different file name extension
> (pi1 pi2 pi3 pc1 pc2 pc3).
> 
> When running running file command version 5.41 on such bitmaps and
> related files i get an output like:
> ARTIS3.PC2:                  data
> BEETHVEN.IMG:                GEM Image data 224 x 131,
> 			     1 planes, 352 x 352 pixelsize
> CHURCH.IMG:                  GEM Image data 224 x 170,
> 			     1 planes, 352 x 352 pixelsize
> Faux-Spitzen.abr:            data
> GAMEOVR4.IMG:                GEM XIMG Image data 256 x 176,
> 			     4 planes, 372 x 372 pixelsize
> GNUCHESS.PC1:                data
> LEREDACT.PI3:                data
> SMTHDRAW.PC3:                data
> TBX_DEMO.PI3:                data
> bigspid.pi1:                 X11 SNF font data, MSB first
> bilboule.pi1:                data
> clinton.img:                 GEM HYPERPAINT Image data 77 x 87,
> 			     4 planes, 338 x 372 pixelsize
> g3test.g3:                   data
> gnucash-4.8.setup.exe.aria2: data
> hpcc88.lif:                  lif file "HPCC88", version 1,
> 			     directory length 12,
> 			     extensions 0x4d00000002...,
> 			     1st file MCITFYP4
> load-v0001.aria2:            data
> medres.pi2:                  data
> msvcrt.lib:                  Intel ia64 COFF object file,
> 			     not stripped, 91 sections,
> 			     symbol offset=0xa8d3, 644 symbols,
> 			     created Tue Sep 19 05:18:08 2006,
> 			     1st section name ".drectve"
> plpbt.iso:                   ISO 9660 CD-ROM filesystem data
> 			     'Plop Boot Manager 5.0.14' (bootable)
> sigirl1.pi3:                 Intel ia64 COFF object file,
> 			     not stripped, 256 sections,
> 			     1st section name ""
> 
> For comparison reason i run the file format identification utility
> TrID ( See https://mark0.net/soft-trid-e.html).
> Most Atari Degas bitmap are described correctly by TrID as
> DEGAS bitmap by first two bytes, sometimes with low recognition rate.
> Only the PI1 examples are not recognised, probably because the
> patterns are too unspecific.
> The PI2 examples are described as "DEGAS med-res bitmap" by
> definition bitmap-pi2-degas.trid.xml. The PI3 examples are described
> as "DEGAS hi-res bitmap" by definition bitmap-pi3-degas.trid.xml.
> The PC1 examples are described as "DEGAS low-res compressed bitmap"
> by bitmap-pc1-degas.trid.xml. The PC2 examples are described as
> "DEGAS med-res compressed bitmap" by bitmap-pc2-degas.trid.xml.
> The PC3 examples are described as "DEGAS hi-res compressed bitmap"
> by bitmap-pc3-degas.trid.xml. (See appended DEGAS-trid-v.txt.gz ).
> 
> Luckily TrID with -v option display correct file name extension and
> also URL pointing to file format specification. That is expressed
> inside Magdir/images by additional comment lines likes:
> # URL:		http://fileformats.archiveteam.org/wiki/DEGAS_image
> # Reference:	https://wiki.multimedia.cx/index.php?title=Degas
> #		http://mark0.net/download/triddefs_xml.7z/defs/b
> #		bitmap-pi2-degas.trid.xml bitmap-pi3-degas.trid.xml
> #		bitmap-pc1-degas.trid.xml bitmap-pc2-degas.trid.xml
> #		bitmap-pc3-degas.trid.xml
> 
> On that site download links for examples and graphic tools are
> mentioned. I verified information by NetPBM tool pi3topbm, XnView
> command line tool nconvert and Deark software.
> 
> Unfortunately there exist no strong significant magic pattern (only 2
> bytes) for such bitmap images. So i put displaying part inside sub
> routine degas-bitmap which starts like:
> 0	name		degas-bitmap
>> 0	ubyte		x		Atari DEGAS
> !:mime	image/x-atari-degas
>> 0	ubyte		=0x80		Elite compressed
>>> 32042	ubequad		x		Elite
>> 0	beshort		0x0000		bitmap
> !:ext	pi1
>> 0	beshort		0x0001		bitmap
> !:ext	pi2
>> 0	beshort		0x0002		bitmap
> !:ext	pi3
>> 0	beshort		0x8000		bitmap
> !:ext	pc1
>> 0	beshort		0x8001		bitmap
> !:ext	pc2
>> 0	beshort		0x8002		bitmap
> !:ext	pc3
>> 1	ubyte		=0		320 x 200 x 16
>> 1	ubyte		=1		640 x 200 x 4
>> 1	ubyte		=2		640 x 400 x 2
> 
> The first byte determinate if the image bytes are compressed. The
> value 80h means compressed variant and 0 means uncompressed.
> The second byte determinate the image resolution and colour depth.
> The value 0 means low resolution (320×200, 16 colours). One means
> medium resolution (640×200, 4 colours) and value two means high
> resolution (640×400, 2 colours). Depending on compression and
> resolution the file name extension varies. The documentation also
> mention a second extension SUH for uncompressed high resolution,
> but i myself found no such examples. Instead of generic mime type
> application/octet-stream i show a user defined one. For the
> uncompressed images exist a "elite" variant. There after the pixel
> data animation information about colour to change, direction and
> time delay is stored. This can also be shown at the end by
> additional debugging lines like:
>> 32034  ubequad	!0	\b, color animations %16.16llx (left)
>>> 32042  ubequad	!0	%16.16llx (right)
>> 32050  ubequad	!0	\b, channel directions %16.16llx
>> 32058  ubequad	!0	\b, channel delays %16.16llx
> 
> After the 2 starting bytes the colour palette is stored til offset
> 34. The Atari ST palette has place for 16 colour entries. Each entry
> occupies 2 bytes ( or 16 bits) in big endian order. Not all 16 bits
> are used. Some examples use only 9 bits ( ?????RRR?GGG?BBB with R for
> red, G for green, B for Blue and ? for unused. That means 512
> different colours) and some use 12 bits ( ????RRRRGGGGBBBB . That
> means 4096 different colours). In documentation also an Atari
> Spectrum 512 Enhanced 15 bit palette is mentioned, but luckily this
> variant apparently does not be used for DEGAS images. According to
> documentation you can not rely that unused bits are zero like in
> mentioned "bad" example bilboule.pi1, but i myself found that in most
> examples unused bits are zero ( so replace ? by 0 in bit mask).
> For control reason show the first colour entries by lines like:
>> 2	ubeshort	x		\b, color palette %4.4x
>> 4	ubeshort	x		%4.4x
>> 6	ubeshort	x		%4.4x
>> 8	ubeshort	x		%4.4x
>> 10	ubeshort	x		%4.4x
> So often we see value 0000. That means black colour. The value 0777
> means white colour in 9 bit variant and 0fff means white in 12 varian
> t.
> So this information can be used as an additional test. So DEGAS
> low-resolution compressed bitmaps (like: BATTLSHP.PC1 GNUCHESS.PC1
> MEDUSABL.PC1 MOONLORD.PC1 WILDROSE.PC1) are recognized by lines like:
> 0		beshort			0x8000
>> 2		ubeshort&0xF000		0
>>> 0		use		degas-bitmap
> 
> The first test for starting two bytes is also true for lif files like
> hpcc88.lif handled by Magdir/lif. But there at offset 2 the volume
> label ( ASCII like HPCC88 or hexadecimal 485043433838 in example
> hpcc88.lif) is stored. So by second test line for unused (that means
> zero) colour palette bits lif examples are skipped and the test
> succeeds for PC1 bitmaps.
> PI1 examples like bigspid.pi1 and bilboule.pi1 are done by lines like
> :
> 0		beshort		0x0000
>> 2		uquad		!0
>>> 4		ubeshort&0xF000	0
>>>> 0		use		degas-bitmap
> Here by second test line some "bad" ISO 9660 CD-ROM filesystems like
> plpbt.iso are skipped. Zero value here would be interpreted as 4
> black colours at the beginning of the colour palette. For real DEGAS
> bitmaps one could find one black colour (that means value 0000) in
> the colour palette, but then the other entries have other colours
> (non zero values) in other entries. So example plpbt.iso with
> 8-byte zero value is skipped.
> 
> For Atari mid-res DEGAS bitmap PI2 examples the lines looks like
> 0		beshort		0x0001
> ...
>>>>>>>> 32026		quad		x
>>>>>>>>> 0	use		degas-bitmap
> After test for weak 2 starting bytes by test line eight the GEM
> HYPERPAINT Image clinton.img is skipped by check for existence of
> bytes at the end of DEGAS images.
> 
> The differentiation between DEGAS PI3 examples and Adobe PhotoShop
> Brush ABR is a little bit tricky. For debugging purpose show my
> observed information about ABR by lines like:
>>> 19		ubyte			!0	\b, NOTE LENGTH %u
>>>> 21		lestring16		x	\b, BRUSH NOTE "%s"
> So for example Faux-Spitzen.abr i get note string "Gitter - klein "
> with length 15. So for example "Verschiedene Spitzen.abr" i get with
> length 8 the note string "Kreis 1 ".
> 
> So if this string length is zero i assume that is not an ABR. So such
> examples must be DEGAS images. So many examples ( like: 4th_ofj2.pi3
> GEMINI03.PI3 PEOPLE18.PI3 POWERFIX.PI3 abydos.pi3 highres.pi3
> sigirl1.pi3 vanna5.pi3) are handled by branch like:
>>> 19		ubyte			=0
>>>> 0		use			degas-bitmap
> Because brush note string is stored as UTF-16 the stored string
> length multiplied with 2 gives the number of bytes occupies by that
> string. So this information can be used to inspect the last
> last character of Adobe PhotoShop Brush UTF16-LE string and
> terminating nil character (that are 4 bytes) by lines like
>>> 19		ubyte			!0
>>>> (19.b*2)	ubequad			x
>>>>> &8		ubelong			x \b, LAST CHAR+NIL %8.8x
> For example "Faux-Spitzen.abr" here i get hexadecimal value 006e0000
> (character n) and for "Verschiedene Spitzen.abr" i get hexadecimal
> value 00310000 (character 1).
> 
> So when test for such nil bytes gives non zero value, it must be
> DEGAS image. So many PI3 examples (like ARABDEMO.PI3 ELMRSESN.PI3
> GEMVIEW.PI3 LEREDACT.PI3 PICCOLO.PI3 REPRO_JR.PI3 ST_TOOLS.PI3
> TBX_DEMO.PI3 evgem7.pi3) are skipped by other branch with additional
> nil test lines like:
>>>>> &8		ubelong&0xff00ffFF	!0
>>>>>> 0		use		degas-bitmap
> 
> If test for such nil bytes gives zero value, test it again but now
> looks if place of last character of note string contains value is
> valid. If this value is "too low" (that means non printable
> character) it must be a DEGAS image. If this value  is "high" enough
> is is a "normal" printable character. That means it is a Photoshop
> ABR. By this last branch the remaining last DEGAS bitmaps ( like
> BASICNES.PI3 DB_HELP.PI3 DB_WRITR.PI3 LEREDACT.PI3) are skipped by
> lines like:
>>>>> &8		ubelong&0xff00ffFF	=0
>>>>>> &-4	ubelong&0x00FF0000	<0x00200000
>>>>>>> 0		use		degas-bitmap
> 
> Some DEGAS high-res uncompressed bitmaps (like GEMINI03.PI3
> MODEM2.PI3 POWERFIX.PI3 sigirl1.pi3 vanna5.pi3) are misidentified as
> "Intel ia64 COFF object file" because the 2 byte start pattern is the
> same. That was expressed by lines inside Magdir/intel like:
> 0	leshort		0x0200
>> 0	use				display-coff
> Luckily the displaying part is done by subroutine display-coff inside
> Magdir/coff. So only additional test lines must be inserted before
> calling sub routine. For all my misidentified PI3 examples the
> interpreted first section name was nil, whereas for real COFF object
> file we get here typical 8-byte sized names (like .text .data
> .debug$S .drectve .testseg). So for most COFF objects the starting
> character is a point character (0x2E). If i remember right some
> Borland compiler for example use DATA instead of .data. So at least
> i assume that starting character is like ASCII printable (that
> means value "high enough"), whereas for DEGAS image at that offset
> the colour palette entry number 10 starts. Because of 4 reserved
> bits (that are in most cases zero) we get here a low value.
> So DEGAS images can be distinguished with very high rate from COFF
> object files. Unfortunately the section name can appear later if COFF
> sample contains an optional header. But in documentation is written
> that COFF object files have not header part compared with COFF
> executables. So i must check by additional second test for F_EXEC
> flag bit. If flag is set, then it is an executable and i can
> call directly sub routine. If this bit is not set, is is an object
> file and i check for starting character of section name befor calling
> sub routine. So magic lines now become like:
> 0	leshort		0x0200
>> 18	leshort		^0x0002
>>> 20	ubyte		>0x1F
>>>> 0	use				display-coff
>> 18	leshort		&0x0002
>>> 0	use				display-coff
> 
> After applying the above mentioned modifications by patches
> file-5.41-images-degas.diff and file-5.41-intel-pi3.diff
> then all my Degas bitmaps are correctly identified and some
> misidetfication vanish like:
> 
> ARTIS3.PC2:                  Atari DEGAS Elite compressed bitmap
> 			     640 x 200 x 4, color palette
> 			     0fff 0f00 00f0 0000 0007 ...
> BEETHVEN.IMG:                GEM Image data 224 x 131,
> 			     1 planes, 352 x 352 pixelsize
> CHURCH.IMG:                  GEM Image data 224 x 170,
> 			     1 planes, 352 x 352 pixelsize
> Faux-Spitzen.abr:            data
> GAMEOVR4.IMG:                GEM XIMG Image data 256 x 176,
> 			     4 planes, 372 x 372 pixelsize
> GNUCHESS.PC1:                Atari DEGAS Elite compressed bitmap
> 			     320 x 200 x 16, color palette
> 			     0221 0000 0310 0420 0530 ...
> LEREDACT.PI3:                Atari DEGAS Elite bitmap
> 			     640 x 400 x 2, color palette
> 			     0fff 0f00 00f0 0000 0fff ...
> SMTHDRAW.PC3:                Atari DEGAS Elite compressed bitmap
> 			     640 x 400 x 2, color palette
> 			     0777 0700 0070 0000 0777 ...
> TBX_DEMO.PI3:                Atari DEGAS Elite bitmap
> 			     640 x 400 x 2, color palette
> 			     0777 0700 0070 0000 0777 ...
> bigspid.pi1:                 Atari DEGAS Elite bitmap
> 			     320 x 200 x 16, color palette
> 			     0004 0025 0037 0000 0410 ...
> bilboule.pi1:                Atari DEGAS Elite bitmap
> 			     320 x 200 x 16, color palette
> 			     0000 0111 8222 0333 fcc4 ...
> clinton.img:                 GEM HYPERPAINT Image data 77 x 87,
> 			     4 planes, 338 x 372 pixelsize
> g3test.g3:                   data
> gnucash-4.8.setup.exe.aria2: data
> hpcc88.lif:                  lif file "HPCC88", version 1,
> 			     directory length 12,
> 			     extensions 0x4d00000002...,
> 			     1st file MCITFYP4
> load-v0001.aria2:            data
> medres.pi2:                  Atari DEGAS Elite bitmap
> 			     640 x 200 x 4, color palette
> 			     0777 0700 0070 0000 0007 ...
> msvcrt.lib:                  Intel ia64 COFF object file,
> 			     not stripped, 91 sections,
> 			     symbol offset=0xa8d3, 644 symbols,
> 			     created Tue Sep 19 05:18:08 2006,
> 			     1st section name ".drectve"
> plpbt.iso:                   ISO 9660 CD-ROM filesystem data
> 			     'Plop Boot Manager 5.0.14' (bootable)
> sigirl1.pi3:                 Atari DEGAS Elite bitmap
> 			     640 x 400 x 2, color palette
> 			     0001 0000 0000 0000 0000 ...
> 
> I hope my diff files can be applied in future version of file utility
> .
> 
> 
> With best wishes
> Jörg Jenderek
> - --
> Jörg Jenderek
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
> 
> iF0EARECAB0WIQS5/qNWKD4ASGOJGL+v8rHJQhrU1gUCYbFWxwAKCRCv8rHJQhrU
> 1rqHAKCpHT9jL3dmki5XL/+jSjpzVHjyYwCfXx5NiYn9nsJ3jwLeoDMKgJCbPqg=
> =Wm0D
> -----END PGP SIGNATURE-----
> <file-5_41-images-degas_diff.DEFANGED-2231><file-5_41-images-degas_diff_sig.DEFANGED-2232><DEGAS-trid-v.txt.gz><file-5_41-intel-pi3_diff.DEFANGED-2233><file-5_41-intel-pi3_diff_sig.DEFANGED-2234>--
> File mailing list
> File at astron.com
> https://mailman.astron.com/mailman/listinfo/file
> <sanitizer.log>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <https://mailman.astron.com/pipermail/file/attachments/20211210/6feaea2a/attachment.asc>


More information about the File mailing list